Python web scraping without browser

0 votes
I am trying to webs scrape a lot of pages. Because the browser is opening the page, it is taking too much time. Is there a way I can scrape the page without opening a browser?
May 9 in Python by Vindo
9 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

Yes, you can scape the web without a browser. You can use the request library to get the html data and then use BeautifulSoup to find elements or extract data. To get html data without opening the browser, you can use the following code:

url = "<Enter the page url here>"
page=requests.get(url)
soup = BeautifulSoup(page.text, 'html.parser')
answered May 9 by John

Related Questions In Python

0 votes
1 answer

How to perform web scraping with python?

Hey, there are various libraries used in ...READ MORE

answered Apr 20, 2018 in Python by aayushi
• 750 points
166 views
+2 votes
11 answers

How to print array/ list without brackets in python?

name=["a","b","c"] print (', '.join(name)) It will simply take all ...READ MORE

answered Apr 18, 2018 in Python by aayushi
• 750 points
12,194 views
0 votes
1 answer

How to exit from Python without traceback?

shutil has many methods you can use. One ...READ MORE

answered May 10, 2018 in Python by charlie_brown
• 7,710 points
80 views
0 votes
1 answer

Best way to create a simple python web service

web.py is probably the simplest web framework ...READ MORE

answered Jul 20, 2018 in Python by Priyaj
• 56,120 points
18 views
0 votes
1 answer

How to parse html file to BeautifulSoup?

Hey. Refer to the following code: driver.get("link") html = ...READ MORE

answered Apr 2 in Python by Kirti
12 views
+2 votes
2 answers

Job roles for Python

Hey, Python Developer responsibilities include writing and testing ...READ MORE

answered May 16 in Python by Gitika
• 8,380 points
29 views
0 votes
1 answer

How to check the version of Python?

you can check the version of python ...READ MORE

answered Sep 25, 2018 in Python by SDeb
• 9,660 points
14 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.