How to web scrape using python without using a browser?

0 votes
I am trying to extract some data from a website. I am using selenium for web scraping. In certain conditions, the code will have to open many tabs which become a mess. Is there any way I can Web scrape without opening a browser?
Apr 2, 2019 in Python by Tina
1,105 views

1 answer to this question.

0 votes

Yes, you can use the headless mode. When you use headless mode, you do use a browser but there is no UI for it. To initialize it, you will need to change some options. Refer to the below command:

from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')  # Last I checked this was necessary.
driver = webdriver.Chrome(CHROMEDRIVER_PATH, chrome_options=options)
answered Apr 2, 2019 by Yogi

Related Questions In Python

0 votes
1 answer

How can I write code to find a palindrome in python without using string functions?

In Logic 1, try if i<int(length/2): instead of if i<int((length/2+1)): In ...READ MORE

answered Jan 16, 2019 in Python by charlie_brown
• 7,740 points
4,699 views
+2 votes
2 answers

How to make a laplacian pyramid using OpenCV python?

down voteacceptTheeThe problem is that you're iterating ...READ MORE

answered Apr 3, 2018 in Python by charlie_brown
• 7,740 points
1,828 views
0 votes
1 answer

How to convert an integer to a string using Python?

Here is an easy solution: def numberToBase(n, b): ...READ MORE

answered Nov 23, 2018 in Python by Nymeria
• 3,520 points

edited Dec 12, 2018 by Nymeria 65 views
0 votes
1 answer

How to get the return value from a thread using python?

You don't need to change your existing ...READ MORE

answered Dec 3, 2018 in Python by Nymeria
• 3,520 points

edited Dec 19, 2018 by Nymeria 19,127 views
0 votes
1 answer

How to parse html file to BeautifulSoup?

Hey. Refer to the following code: driver.get("link") html = ...READ MORE

answered Apr 2, 2019 in Python by Kirti
92 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

How to download a file over HTTP using Python?

In Python 2, use urllib2 which comes ...READ MORE

answered Oct 18, 2018 in Python by SDeb
• 13,230 points
79 views
0 votes
1 answer

How to put a variable inside a String using Python?

In the easiest way, you can create ...READ MORE

answered Nov 23, 2018 in Python by Nymeria
• 3,520 points

edited Dec 12, 2018 by Nymeria 66 views