Python web scraping without browser

0 votes
I am trying to webs scrape a lot of pages. Because the browser is opening the page, it is taking too much time. Is there a way I can scrape the page without opening a browser?
May 9, 2019 in Python by Vindo
1,502 views

1 answer to this question.

0 votes

Yes, you can scape the web without a browser. You can use the request library to get the html data and then use BeautifulSoup to find elements or extract data. To get html data without opening the browser, you can use the following code:

url = "<Enter the page url here>"
page=requests.get(url)
soup = BeautifulSoup(page.text, 'html.parser')
answered May 9, 2019 by John

Related Questions In Python

0 votes
1 answer

How to perform web scraping with python?

Hey, there are various libraries used in ...READ MORE

answered Apr 20, 2018 in Python by aayushi
• 750 points
976 views
0 votes
1 answer
0 votes
1 answer

I want to download a file from the website by web scraping. Can anyone explain how to do this in jupyter lab (python) with an example?

Hey, Web scraping is a technique to automatically ...READ MORE

answered Apr 7, 2020 in Python by Gitika
• 65,870 points
669 views
0 votes
1 answer

web scraping using python

Hey, @Pythonist, You can try this, ##################################### Method 1 import ...READ MORE

answered May 6, 2020 in Python by Gitika
• 65,870 points
273 views
0 votes
1 answer
0 votes
1 answer

How to parse html file to BeautifulSoup?

Hey. Refer to the following code: driver.get("link") html = ...READ MORE

answered Apr 2, 2019 in Python by Kirti
956 views
0 votes
1 answer
0 votes
1 answer

how to check the syntax of python program without running the program?

Yes, it is possible to check the ...READ MORE

answered Jun 27, 2019 in Python by Arvind
• 3,000 points
1,438 views