麻烦点击下一页的按钮 [英] Trouble clicking on the button for the next page

问题描述

我已经用python结合硒编写了一些代码.我打算从网页解析表.我已经工作了.但是，当我尝试单击下一页按钮时会遇到麻烦.刮板仅从第一页分析表，而不单击下一个按钮，它退出时不会引发任何错误.所以，我无法理解我所缺少的.

I've written some code in python in combination with selenium. I intended to parse the table from a webpage. I've got it working. However, trouble comes up when i try to click on the next page button. The scraper only parse the table from the first page and instead of clicking the next button it quits without throwing any error. So, i can't understand what I'm missing.

以下是完整的代码供您考虑:

Here is the full code for your consideration:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)

driver.get("https://toolkit.financialexpress.net/santanderam")

wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'table.fe-datatable')))

tab_data = driver.find_element_by_css_selector('table.fe-datatable')

while True:
    wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'tr')))
    list_rows = [[cell.text for cell in row.find_elements_by_css_selector('td')]
                 for row in tab_data.find_elements_by_css_selector('tr')]
    for data in list_rows:
        print(data)

    try:
        driver.find_element_by_css_selector('a.ui-paging-next').click()
    except:
        break

driver.quit()

存在下一页按钮的元素:

Elements within which the next-page button exists:

<div class="pagination ui-widget"><span class="ui-paging-current ui-state-default ui-state-disabled ui-corner-all ui-paging-prev">Prev</span><span class="ui-paging-current ui-state-default ui-state-disabled ui-state-highlight ui-corner-all">1</span><a class="ui-paging-button ui-state-default ui-corner-all" href="#">2</a><a class="ui-paging-button ui-state-default ui-corner-all" href="#">3</a><a class="ui-paging-button ui-state-default ui-corner-all" href="#">4</a><span class="ui-state-default ui-corner-all ui-state-disabled ui-paging-ellipse">...</span><a class="ui-paging-button ui-state-default ui-corner-all ep" href="#">7</a><a class="ui-paging-button ui-state-default ui-corner-all ui-paging-next" href="#">Next</a></div>

推荐答案

@Grasshopper已经提供了一个解决方案，但是我将尝试为您提供更多详细信息，以了解您的代码为何失败

@Grasshopper has already provided with a solution, but I'll try to give more details for you to understand why your code fails

页面源中存在两个具有相同HTML代码的链接:第一个链接是隐藏的，第二个链接(您需要的链接)没有.

There are two links with the same HTML code present in page source: the first is hidden, second (the one that you need) is not.

您可以使用

print(len(driver.find_elements_by_css_selector('a.ui-paging-next')))

虽然css-selector或XPath仅返回您第一次出现的情况，但按链接文本搜索仅返回具有可见文本的链接:

While css-selector or XPath returns you simply the first occurence, search by link text returns link with the visible text only:

print(len(driver.find_elements_by_link_text('Next')))

这就是为什么您的find_element_by_css_selector(...)代码不起作用，而find_element_by_link_text(...)起作用的原因.

That's why your find_element_by_css_selector(...) code doesn't work, but find_element_by_link_text(...) does.

还要注意那一行

wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'table.fe-datatable')))

应该已经返回了您所需的元素，所以在

should already return you required element, so there is no need in

tab_data = driver.find_element_by_css_selector('table.fe-datatable')

只需使用

tab_data = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'table.fe-datatable')))

为避免获取StaleElementReferenceException，您应在每次迭代中重新定义tab_data，因为首页上定义的tab_data在下一页将无法访问.只需将tab_data定义放入while循环

To avoid getting StaleElementReferenceException you should re-define your tab_data on each iterarion as tab_data defined on first page will not be accessible on the next page. Just put tab_data definition inside the while loop

更新

在您的代码中尝试替换

try:
    driver.find_element_by_link_text('Next').click()
except:
    break

使用

first_row = driver.find_element_by_css_selector('table.fe-datatable tr.odd').text
try:
    driver.find_element_by_link_text('Next').click()
except:
    break
wait.until(lambda driver: driver.find_element_by_css_selector('table.fe-datatable tr.odd').text != first_row)

这篇关于麻烦点击下一页的按钮的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！