scrape download file having customize selection using python selenium - stock screener site

0 votes

Hi,

I couldn't figure out how to paste clipboard image of the screenshot of the page nor upload the image in this query.

I would like to use Selenium and Python to download file. The thing is that there are selections that can be checked for the query before clicking on the "Download" button.

1) a set of checkbox with range bar of selection criterion e.g. Market Cap, 52 weeks % change.

2) a list of customize options (a dropdown list to display which column fields you want to see) that can be checked.

I have no idea how to code this. The code examples I managed to find teaches how to download a file but I couldn't find examples on how to download a file that has customize options.

The website is https://www2.sgx.com/securities/stock-screener

I'm grateful for your time in reading my query and appreciate if you could help me out or point me to some similar examples that I can follow.

Thank you.

May 9, 2019 in Selenium by Yi Wen Edwin
• 150 points
2,555 views

1 answer to this question.

0 votes
Best answer

Hi YI Wen Edwin, I am glad that I could help you with your issue. I am sharing the code snippet which would resolve your query on how to customize the filters and then download the stocks file:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time

driver = webdriver.Chrome("C:\\\\Users\\\\Abha_Rathour\\\\Downloads\\\\ExtractedFiles\\\\chromedriver_win32\\\\chromedriver.exe")
driver.maximize_window()
driver.implicitly_wait(2)
driver.get("https://www2.sgx.com/securities/stock-screener")

driver.find_element_by_xpath('//*[@id="beta-warning-dialog"]/div[2]/div/div/div/sgx-input-checkbox/label/span').click()
driver.find_element_by_xpath('//*[@id="beta-warning-dialog"]/div[2]/div/div/div/div[3]/button').click()
driver.find_element_by_xpath('//*[@id="gdpr-banner"]/div/button').click()

time.sleep(5)

driver.find_element_by_xpath("//button[@class='widget-stock-screener-button widget-stock-screener-filters-toggle']").click()
driver.find_element_by_xpath("//input[@name='sector']").click()
driver.find_element_by_xpath("//span[contains(text(),'Energy')]").click()

driver.find_element_by_xpath("//span[contains(text(),'Tot. Rev ($M)')]").click()
driver.find_element_by_xpath("//span[contains(text(),'Mkt Cap ($M)')]").click()
driver.find_element_by_xpath("//span[contains(text(),'52-wk %Pr. Chg.')]").click()
driver.find_element_by_xpath("//span[contains(text(),'Net Profit %')]").click()

driver.find_element_by_xpath("//span[contains(text(),'Download')]").click()

time.sleep(3)

driver.close()


Now, read the following steps to understand the code better:

1. First import the ChromeDriver and use some of its basic methods like maximize_window() or implicitly_wait(time_in_seconds) to setup the driver properties.

2. Then to close the pop-up which shows every-time you open the page, write first 3 lines of driver.find_element... code.

3. Now, use driver.find_element_by_xpath to click on Select Sector dropdown and Show More Filters. You can use this method also for selecting which Sector you want to choose. Eg. I chose Energy sector in my code.

4. Next, use same driver.find_element_by_xpath method individually to select all the Selection Criteria you wish to choose.

5. Finally write the driver method for downloading the file.

Note: I used time.sleep(time) at 2 places so that page could all the elements properly and you could see the automation process.

answered May 9, 2019 by Abha
• 28,140 points

selected May 9, 2019 by Abha

Related Questions In Selenium

+1 vote
1 answer
0 votes
1 answer

Need to download a file using Selenium

Pretty simple...Find the element or link using ...READ MORE

answered Mar 30, 2018 in Selenium by nsv999
• 5,500 points
3,514 views
0 votes
1 answer

How can I download the *.jar file from http:// seleniumhq.org using selenium WebDriver?

For Selenium Standalone Server use this: profile.setPreference("browser.helperApps.neverAsk.saveToDisk", "application/java-archive"); and ...READ MORE

answered Apr 9, 2018 in Selenium by Martin
• 4,320 points
3,574 views
0 votes
1 answer
0 votes
2 answers

Finding WebDriver element with Class Name in java

The better way to handle this element ...READ MORE

answered Apr 10, 2018 in Selenium by nsv999
• 5,500 points
13,767 views
0 votes
2 answers

Problem while using InternetExplorerDriver in Selenium WebDriver

enable trusted connection  in internet explorer by ...READ MORE

answered Aug 31, 2020 in Selenium by Sri
• 3,190 points
9,186 views
0 votes
1 answer

Geo-location microphone camera pop up

To Allow or Block the notification, access using Selenium and you have to ...READ MORE

answered May 11, 2018 in Selenium by Samarpit
• 5,910 points
7,261 views
0 votes
2 answers

How to use such xpath to find web elements

xpath are two types. 1) Absolute XPath:    /html/b ...READ MORE

answered Sep 3, 2020 in Selenium by Sri
• 3,190 points
7,901 views
0 votes
1 answer

Not able to upload a file in Selenium Webdriver using python Scripting

Hey Srinivas, you can checkout this code ...READ MORE

answered Sep 10, 2019 in Selenium by Abha
• 28,140 points
4,212 views
+1 vote
1 answer

Is it possible to write data into an excel file using Selenium Webdriver?

Hi Simran, if you want to directly ...READ MORE

answered May 8, 2019 in Selenium by Abha
• 28,140 points
6,750 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP