用Python中的Selenium停止动态页面中的滚动 [英] Stop the Scroll in Dynamic Page with Selenium in Python

问题描述

大家好，我正尝试使用硒和scrapy来从提取一些信息https://answers.yahoo.com/dir/index/discover?sid=396545663

我尝试了不同的方法，我使用Selenium并将PhantomJs设置为驱动程序。
对于向下滚动页面，这是一个无限滚动页面，我使用这条指令：

pre $ elem.send_keys（Keys .PAGE_DOWN）

模拟按下Page Down按钮，而不是JavaScript函数：

  browser.execute_script（window.scrollTo（0，document.body.scrollHeight）;）
  

因为这个看起来会在页面中加载更少的元素。

问题是我如何知道我何时到达页面底部？是无限滚动页面，所以我不知道什么时候结束我需要向下滚动，但我没有任何元素在底部分析。

实际上我使用的是临时循环，但看起来真的很蠢。

谢谢 解决方案

我真的会寻找那个加载...指标。 等待在每次滚动时都可见，但如果您得到一个 TimeoutException - 这次没有加载指示器，并且没有更多的项目需要加载。

示例实现：

  from selenium.webdriver.common.by import通过
 from selenium.webdriver.support.ui import WebDriverWait $来自selenium.webdriver.support的b $ b导入expected_conditions作为EC 
 
等待= WebDriverWait（驱动程序，10）
 
而真：
＃滚动
 browser.execute_script（window.scrollTo（0，document.body.scrollHeight）;）
 
 try：
 wait.until（EC.visibility_of_element_located（（By.XPATH， 
除了TimeoutException：
 break＃没有更多帖子被加载 - 退出循环
  

未经测试。

Hello everyone i'm trying to use selenium and scrapy to scraping some information from https://answers.yahoo.com/dir/index/discover?sid=396545663

I try different method, i use Selenium and setting PhantomJs like driver. For scrolling down the page, it's a infinite scroll page, i use this instruction:

elem.send_keys(Keys.PAGE_DOWN)

For simulating the press of Page Down button, instead of the JavaScript function:

browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")

Because this one "seems" load less elements in the page.

The main problem is how i can know when i have reached the bottom of the page? Is "Infinite Scroll" page so i can't know when it end i need to scroll down, but i don't have any element in the bottom to analyze.

Actually i use temporized cycle, but look really stupid.

Thanks

解决方案

I would actually look for that "Loading..." indicator. Wait for it to be visible on every scroll, but if you'll get a TimeoutException - there was no loading indicator this time and there are no more items to load.

Sample implementation:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

wait = WebDriverWait(driver, 10)

while True:
    # do the scrolling
    browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")

    try:
        wait.until(EC.visibility_of_element_located((By.XPATH, "//*[. = 'Loading...']")))
    except TimeoutException:
        break  # not more posts were loaded - exit the loop

Not tested.

这篇关于用Python中的Selenium停止动态页面中的滚动的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！