用硒刮屏幕8 [英] Screen scraping with selenium 8

问题描述

这是我的代码：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

browser = webdriver.PhantomJS()
browser.set_window_size(1120, 550)
browser.get("http://www.jamiiforums.com/kenyan-news/225589-kenyan-and-tanzanian-surburbs.html")

username = browser.find_element_by_id("navbar_username")
password = browser.find_element_by_name("vb_login_password_hint")

username.send_keys("user")
password.send_keys("password")

browser.find_element_by_class_name("loginbutton").click()

wait = WebDriverWait(browser, 10)
wait.until(EC.visibility_of_element_located((By.XPATH, '//h2[contains(., "Redirecting")]')))
wait.until(EC.title_contains('Kenyan & Tanzanian'))


link = browser.find_element_by_xpath('//div[@class="vbseo_liked"]/a[contains(@onclick, "return vbseoui.others_click(this)")]')
link.click()
browser.save_screenshot('screenie.png')
print 'success!!'
browser.close()

For此HTML代码：

For this HTML code:

<div class="vbseo_liked">
<a href="http://www.jamiiforums.com/member.php?u=8355" rel="nofollow">Nyaralego</a>
,
<a href="http://www.jamiiforums.com/member.php?u=8870" rel="nofollow">Sikonge</a>
,
<a href="http://www.jamiiforums.com/member.php?u=8979" rel="nofollow">Ab-Titchaz</a>
and
<a onclick="return vbseoui.others_click(this)" href="http://www.jamiiforums.com/kenyan-news/225589-kenyan-and-tanzanian-surburbs.html#">11 others</a>
like this.
</div>

我希望能够点击此链接：

I want to be able to click on this link:

 <a onclick="return vbseoui.others_click(this)" href="http://www.jamiiforums.com/kenyan-news/225589-kenyan-and-tanzanian-surburbs.html#">11 others</a>

然后在页面被点击后截取页面截图。当我运行代码时，我总是收到这个错误。

And then take a screenshot of the page after it has been clicked. This error I keep getting though when i run the code.

selenium.common.exceptions.NoSuchElementException：消息：{errorMessage：无法找到类名为vbseo_liked的元素

selenium.common.exceptions.NoSuchElementException: Message: {"errorMessage":"Unable to find element with class name 'vbseo_liked'"

推荐答案

在点击之前，等待文章列表加载：

You need to wait for the list of posts to load before making a click:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

browser = webdriver.PhantomJS()
browser.maximize_window()
browser.get("http://www.jamiiforums.com/kenyan-news/225589-kenyan-and-tanzanian-surburbs.html")

username = browser.find_element_by_id("navbar_username")
password = browser.find_element_by_name("vb_login_password_hint")

username.send_keys("username")
password.send_keys("password")

browser.find_element_by_class_name("loginbutton").click()

wait = WebDriverWait(browser, 10)
wait.until(EC.visibility_of_element_located((By.XPATH, '//h2[contains(., "Redirecting")]')))
wait.until(EC.title_contains('Kenyan & Tanzanian'))
wait.until(EC.visibility_of_element_located((By.ID, 'postlist')))

link = browser.find_element_by_xpath('//div[@class="vbseo_liked"]/a[contains(@onclick, "return vbseoui.others_click(this)")]')
link.click()

browser.save_screenshot('screenie.png')
print 'success!!'
browser.close()

注意生成的屏幕截图会非常大（磁盘上大约为39 MB）。

Note that the generated screenshot would be very large (about 39 MB on disk).

这篇关于用硒刮屏幕8的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！