Python web scraping without browser

0 votes
I am trying to webs scrape a lot of pages. Because the browser is opening the page, it is taking too much time. Is there a way I can scrape the page without opening a browser?
May 9 in Python by Vindo
38 views

1 answer to this question.

0 votes

Yes, you can scape the web without a browser. You can use the request library to get the html data and then use BeautifulSoup to find elements or extract data. To get html data without opening the browser, you can use the following code:

url = "<Enter the page url here>"
page=requests.get(url)
soup = BeautifulSoup(page.text, 'html.parser')
answered May 9 by John

Related Questions In Python

0 votes
1 answer

How to perform web scraping with python?

Hey, there are various libraries used in ...READ MORE

answered Apr 20, 2018 in Python by aayushi
• 750 points
278 views
+2 votes
11 answers
0 votes
1 answer

How to exit from Python without traceback?

shutil has many methods you can use. One ...READ MORE

answered May 10, 2018 in Python by charlie_brown
• 7,720 points
113 views
0 votes
1 answer

How to parse html file to BeautifulSoup?

Hey. Refer to the following code: driver.get("link") html = ...READ MORE

answered Apr 2 in Python by Kirti
38 views
0 votes
1 answer

how to check the syntax of python program without running the program?

Yes, it is possible to check the ...READ MORE

answered Jun 27 in Python by Arvind
• 2,960 points
37 views
+2 votes
2 answers

Job roles for Python

Hey, Python Developer responsibilities include writing and testing ...READ MORE

answered May 16 in Python by Gitika
• 25,340 points
43 views