Python web scraping without browser

0 votes
I am trying to webs scrape a lot of pages. Because the browser is opening the page, it is taking too much time. Is there a way I can scrape the page without opening a browser?
May 9 in Python by Vindo
49 views

1 answer to this question.

0 votes

Yes, you can scape the web without a browser. You can use the request library to get the html data and then use BeautifulSoup to find elements or extract data. To get html data without opening the browser, you can use the following code:

url = "<Enter the page url here>"
page=requests.get(url)
soup = BeautifulSoup(page.text, 'html.parser')
answered May 9 by John

Related Questions In Python

0 votes
1 answer

How to perform web scraping with python?

Hey, there are various libraries used in ...READ MORE

answered Apr 20, 2018 in Python by aayushi
• 750 points
317 views
+2 votes
11 answers
0 votes
1 answer

How to exit from Python without traceback?

shutil has many methods you can use. One ...READ MORE

answered May 10, 2018 in Python by charlie_brown
• 7,720 points
126 views
0 votes
1 answer

How to parse html file to BeautifulSoup?

Hey. Refer to the following code: driver.get("link") html = ...READ MORE

answered Apr 2 in Python by Kirti
54 views
0 votes
1 answer

how to check the syntax of python program without running the program?

Yes, it is possible to check the ...READ MORE

answered Jun 27 in Python by Arvind
• 2,960 points
44 views