How to web scrape using python without using a browser

0 votes
I am trying to extract some data from a website. I am using selenium for web scraping. In certain conditions, the code will have to open many tabs which become a mess. Is there any way I can Web scrape without opening a browser?
Apr 2, 2019 in Python by Tina
10,332 views

1 answer to this question.

+1 vote

Yes, you can use the headless mode. When you use headless mode, you do use a browser but there is no UI for it. To initialize it, you will need to change some options. Refer to the below command:

from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')  # Last I checked this was necessary.
driver = webdriver.Chrome(CHROMEDRIVER_PATH, chrome_options=options)


This should work.

To know more, It's recommended to join our Python Training in Chennai today.

answered Apr 2, 2019 by Yogi

edited Oct 7, 2021 by Sarfaraz
it is not working.every time i run code browser opens.
Hi, @Jay,

Could you please elaborate on what you tried to do and what kind of problem you are facing?

Related Questions In Python

0 votes
1 answer

how to send post request to a webpage using cookies and headers in python to scrape the webpage ?

Hey @Avinash, Regarding your query, I would suggest ...READ MORE

answered Sep 29, 2020 in Python by Roshni
• 10,500 points
1,278 views
+2 votes
2 answers

How to make a laplacian pyramid using OpenCV python?

down voteacceptTheeThe problem is that you're iterating ...READ MORE

answered Apr 3, 2018 in Python by charlie_brown
• 7,710 points
3,755 views
0 votes
1 answer

How to convert an integer to a string using Python?

Here is an easy solution: def numberToBase(n, b): ...READ MORE

answered Nov 23, 2018 in Python by Nymeria
• 3,540 points

edited Dec 12, 2018 by Nymeria 367 views
0 votes
1 answer

How to parse html file to BeautifulSoup?

Hey. Refer to the following code: driver.get("link") html = ...READ MORE

answered Apr 2, 2019 in Python by Kirti
1,166 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

How to download a file over HTTP using Python?

In Python 2, use urllib2 which comes ...READ MORE

answered Oct 18, 2018 in Python by SDeb
• 13,300 points
333 views
0 votes
1 answer

How to put a variable inside a String using Python?

In the easiest way, you can create ...READ MORE

answered Nov 23, 2018 in Python by Nymeria
• 3,540 points

edited Dec 12, 2018 by Nymeria 377 views
webinar REGISTER FOR FREE WEBINAR X
Send OTP
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP