Python web scraping without browser

0 votes
I am trying to webs scrape a lot of pages. Because the browser is opening the page, it is taking too much time. Is there a way I can scrape the page without opening a browser?
May 9, 2019 in Python by Vindo
294 views

1 answer to this question.

0 votes

Yes, you can scape the web without a browser. You can use the request library to get the html data and then use BeautifulSoup to find elements or extract data. To get html data without opening the browser, you can use the following code:

url = "<Enter the page url here>"
page=requests.get(url)
soup = BeautifulSoup(page.text, 'html.parser')
answered May 9, 2019 by John

Related Questions In Python

0 votes
1 answer

How to perform web scraping with python?

Hey, there are various libraries used in ...READ MORE

answered Apr 20, 2018 in Python by aayushi
• 750 points
547 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

web scraping using python

Hey, @Pythonist, You can try this, ##################################### Method 1 import ...READ MORE

answered May 6 in Python by Gitika
• 29,490 points
50 views
0 votes
1 answer
0 votes
1 answer

How to parse html file to BeautifulSoup?

Hey. Refer to the following code: driver.get("link") html = ...READ MORE

answered Apr 2, 2019 in Python by Kirti
313 views
0 votes
1 answer
0 votes
1 answer

how to check the syntax of python program without running the program?

Yes, it is possible to check the ...READ MORE

answered Jun 27, 2019 in Python by Arvind
• 2,980 points
166 views