Is it possible to scroll down in a webpage using selenium webdriver programmed on python

Question

I'm actually using WebDriver to parse through page displaying all names of friends on facebook, and i'm extracting all their ids from the AJAX script. But I'm unable to scroll down to get more friends. How to scroll down using Selenium code? I'm using Python.

Answer 1

Yup..you can. This is how its done.

driver.execute_script("window.scrollTo(0, Y)") 

where Y is the height (on a fullhd monitor it's 1080).

Or, you can also use this to scroll down to the bottom of the page.

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

Comments

Not Working Tell me another command

---

Hi Shyamal, can you please state what exactly is causing the problem with above command or what is the error you are facing. You can use this line of code also:

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

Answer 2

I using next code for facebook for group scraping:

        # Pause time after scrool
        SCROLL_PAUSE_TIME = 2

        # Get page height
        last_height = browser.script(query="return document.body.scrollHeight")

        while True:
            # Make scroll down to bottom
            browser.script(query="window.scrollTo(0, document.body.scrollHeight);")

            # Wait to load page
            time.sleep(SCROLL_PAUSE_TIME)
            # Calculate new scroll height and compare with last scroll height
            new_height = browser.script(query="return document.body.scrollHeight")
            # break by end of results tag (<div class="phm _64f">End of Results</div>)
            try:
                browser.find(method="xpath", query="//div[@class='phm _64f'][contains(.,'End of Results')]", error=0)
            except NoSuchElementException:
                pass
            else:
                break
            # break by size
            if new_height == last_height:
                break
            last_height = new_height

Comments

Hi @Mslavikas, can you suggest any shorter way of doing like this:

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

 As this single line of code can be used to scroll a page down.

Your suggestions are welcomed.