How to web scrape using python without using a browser?

0 votes
I am trying to extract some data from a website. I am using selenium for web scraping. In certain conditions, the code will have to open many tabs which become a mess. Is there any way I can Web scrape without opening a browser?
Apr 2 in Python by Tina
80 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

Yes, you can use the headless mode. When you use headless mode, you do use a browser but there is no UI for it. To initialize it, you will need to change some options. Refer to the below command:

from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')  # Last I checked this was necessary.
driver = webdriver.Chrome(CHROMEDRIVER_PATH, chrome_options=options)
answered Apr 2 by Yogi

Related Questions In Python

0 votes
1 answer
+2 votes
2 answers

How to make a laplacian pyramid using OpenCV python?

down voteacceptTheeThe problem is that you're iterating ...READ MORE

answered Apr 3, 2018 in Python by charlie_brown
• 7,710 points
1,005 views
0 votes
1 answer

How to convert an integer to a string using Python?

Here is an easy solution: def numberToBase(n, b): ...READ MORE

answered Nov 23, 2018 in Python by Nymeria
• 3,500 points

edited Dec 12, 2018 by Nymeria 28 views
0 votes
1 answer

How to get the return value from a thread using python?

You don't need to change your existing ...READ MORE

answered Dec 3, 2018 in Python by Nymeria
• 3,500 points

edited Dec 19, 2018 by Nymeria 4,945 views
0 votes
1 answer

How to parse html file to BeautifulSoup?

Hey. Refer to the following code: driver.get("link") html = ...READ MORE

answered Apr 2 in Python by Kirti
15 views
0 votes
1 answer
0 votes
1 answer

How to download a file over HTTP using Python?

In Python 2, use urllib2 which comes ...READ MORE

answered Oct 18, 2018 in Python by SDeb
• 10,700 points
20 views
0 votes
1 answer

How to put a variable inside a String using Python?

In the easiest way, you can create ...READ MORE

answered Nov 23, 2018 in Python by Nymeria
• 3,500 points

edited Dec 12, 2018 by Nymeria 30 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.