Web scraping problem

0 votes
Why do I get errors while performing web scraping? At times I get partial outputs and at times I don't. I do not understand the issue
Jul 31 in Python by Fata
• 1,000 points
15 views

1 answer to this question.

0 votes

This is because many websites do not allow web scraping. Before you start working on any website just check for the status code value. The value needs to be 200. Any number other than this means that the website either disallows web scraping or allows partial scraping.

EXAMPLE:

import requests
from bs4 import BeautifulSoup 
g=requests.get('https://ca.finance.yahoo.com/q?s=AAPL&ql=0')
g.status_code

OUTPUT: 200

answered Jul 31 by Wajiha
• 1,840 points

Related Questions In Python

0 votes
1 answer
0 votes
1 answer

Best way to create a simple python web service

web.py is probably the simplest web framework ...READ MORE

answered Jul 20, 2018 in Python by Priyaj
• 56,520 points
31 views
0 votes
1 answer
0 votes
1 answer

Problem with module xlsxwriter

Even if it looks like the module ...READ MORE

answered Jan 22 in Python by charlie_brown
• 7,720 points
102 views
0 votes
1 answer

How to perform web scraping with python?

Hey, there are various libraries used in ...READ MORE

answered Apr 20, 2018 in Python by aayushi
• 750 points
234 views
0 votes
1 answer
0 votes
1 answer

How to use BeautifulSoup for Webscraping

Your code is good until you get ...READ MORE

answered Sep 6, 2018 in Python by Priyaj
• 56,520 points
192 views
0 votes
1 answer

How to download intext images with beautiful soup

Try this: html_data = """ <td colspan="3"><b>"Assemble under ...READ MORE

answered Sep 10, 2018 in Python by Priyaj
• 56,520 points
497 views
0 votes
1 answer

How to check if a website allows web scraping?

To check if a website allows web ...READ MORE

answered Jun 14 in Python by Wajiha
• 1,840 points
35 views