scrape/download file having customize selection using python selenium - stock screener site

0 votes

Hi,

I couldn't figure out how to paste clipboard image of the screenshot of the page nor upload the image in this query.

I would like to use Selenium and Python to download file. The thing is that there are selections that can be checked for the query before clicking on the "Download" button.

1) a set of checkbox with range bar of selection criterion e.g. Market Cap, 52 weeks % change.

2) a list of customize options (a dropdown list to display which column fields you want to see) that can be checked.

I have no idea how to code this. The code examples I managed to find teaches how to download a file but I couldn't find examples on how to download a file that has customize options.

The website is https://www2.sgx.com/securities/stock-screener

I'm grateful for your time in reading my query and appreciate if you could help me out or point me to some similar examples that I can follow.

Thank you.

May 9 in Selenium by Yi Wen Edwin
• 150 points
305 views

1 answer to this question.

0 votes
Best answer

Hi YI Wen Edwin, I am glad that I could help you with your issue. I am sharing the code snippet which would resolve your query on how to customize the filters and then download the stocks file:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time

driver = webdriver.Chrome("C:\\\\Users\\\\Abha_Rathour\\\\Downloads\\\\ExtractedFiles\\\\chromedriver_win32\\\\chromedriver.exe")
driver.maximize_window()
driver.implicitly_wait(2)
driver.get("https://www2.sgx.com/securities/stock-screener")

driver.find_element_by_xpath('//*[@id="beta-warning-dialog"]/div[2]/div/div/div/sgx-input-checkbox/label/span').click()
driver.find_element_by_xpath('//*[@id="beta-warning-dialog"]/div[2]/div/div/div/div[3]/button').click()
driver.find_element_by_xpath('//*[@id="gdpr-banner"]/div/button').click()

time.sleep(5)

driver.find_element_by_xpath("//button[@class='widget-stock-screener-button widget-stock-screener-filters-toggle']").click()
driver.find_element_by_xpath("//input[@name='sector']").click()
driver.find_element_by_xpath("//span[contains(text(),'Energy')]").click()

driver.find_element_by_xpath("//span[contains(text(),'Tot. Rev ($M)')]").click()
driver.find_element_by_xpath("//span[contains(text(),'Mkt Cap ($M)')]").click()
driver.find_element_by_xpath("//span[contains(text(),'52-wk %Pr. Chg.')]").click()
driver.find_element_by_xpath("//span[contains(text(),'Net Profit %')]").click()

driver.find_element_by_xpath("//span[contains(text(),'Download')]").click()

time.sleep(3)

driver.close()


Now, read the following steps to understand the code better:

1. First import the ChromeDriver and use some of its basic methods like maximize_window() or implicitly_wait(time_in_seconds) to setup the driver properties.

2. Then to close the pop-up which shows every-time you open the page, write first 3 lines of driver.find_element... code.

3. Now, use driver.find_element_by_xpath to click on Select Sector dropdown and Show More Filters. You can use this method also for selecting which Sector you want to choose. Eg. I chose Energy sector in my code.

4. Next, use same driver.find_element_by_xpath method individually to select all the Selection Criteria you wish to choose.

5. Finally write the driver method for downloading the file.

Note: I used time.sleep(time) at 2 places so that page could all the elements properly and you could see the automation process.

answered May 9 by Abha
• 27,180 points

selected May 9 by Abha

Related Questions In Selenium

+1 vote
1 answer
0 votes
1 answer

Need to download a file using Selenium

Pretty simple...Find the element or link using ...READ MORE

answered Mar 29, 2018 in Selenium by nsv999
• 5,110 points
555 views
0 votes
1 answer

How can I download the *.jar file from http:// seleniumhq.org using selenium WebDriver?

For Selenium Standalone Server use this: profile.setPreference("browser.helperApps.neverAsk.saveToDisk", "application/java-archive"); and ...READ MORE

answered Apr 9, 2018 in Selenium by Martin
• 4,220 points
619 views
0 votes
1 answer
0 votes
1 answer

Finding WebDriver element with Class Name in java

The better way to handle this element ...READ MORE

answered Apr 10, 2018 in Selenium by nsv999
• 5,110 points
992 views
0 votes
1 answer
0 votes
1 answer

Geo-location microphone camera pop up

To Allow or Block the notification, access using Selenium and you have to ...READ MORE

answered May 11, 2018 in Selenium by Samarpit
• 5,130 points
1,143 views
0 votes
1 answer

How to use such xpath to find web elements

It's much simpler to use: #MyDiv123 > div.super or ...READ MORE

answered Jun 14, 2018 in Selenium by Samarpit
• 5,130 points
888 views
0 votes
1 answer
+1 vote
1 answer