如何使用 Selenium - Python 检索 WebElement 的文本 [英] How to retrieve the text of a WebElement using Selenium - Python

查看:46
本文介绍了如何使用 Selenium - Python 检索 WebElement 的文本的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我是 Python 和 Web Scraping 的新手,所以请耐心等待.我一直在尝试构建一个网页抓取工具来打开网页、登录和检索某个值.到目前为止,我已经能够打开网页并登录.但是,我根本找不到一种方法来检索(打印)我需要的值.这是我当前代码的样子:

I am new to Python and Web Scraping so please bear with me. I have been trying to build a web scraping tool to open a web page, log-in, and retrieve a certain value. Thus far, I have been able to open the web page and log-in. However, I simply cannot find a way to retrieve (print) the value that I require. This is what my current code looks like:

from selenium import webdriver
from bs4 import BeautifulSoup

driver = webdriver.Chrome(executable_path=r'C:/Users/User/Downloads/chromedriver.exe')

url = "xxxxxxxx"
driver.get(url)
driver.find_element_by_name("username").send_keys("xxxxx")
driver.find_element_by_name("password").send_keys("xxxxx")
elem = driver.find_element_by_css_selector("form#frmMain > a:nth-child(4)")
elem.click()

html = '''<p class="value noWrap" data-bind="text: MarketValue">R 4 516 469.32</p>'''
soup = BeautifulSoup(html, 'lxml')

for p in soup.find_all('p'):
    print(p.string)

driver.quit()

我需要的值嵌入在R 4 516 469.32"上方的 html 变量中.但是,该值每天都在变化.我曾尝试使用 xpath 和 css,但由于某些奇怪的原因,相关值似乎被隐藏了.如何动态引用元素以便能够每天检索新值?

The value I require is embedded in the html variable above "R 4 516 469.32". However, this value changes on a daily basis. I have tried using xpath and css, but the value in question seems to be hidden for some odd reason. How can I refer to the element dynamically in order to be able to retrieve the new value every day?

请注意:我已将网址清空,因为这是用于公司目的的网站.

Please note: I have blanked out the url as this is a website used for company purposes.

请帮忙!

非常感谢

推荐答案

所需元素是动态元素,因此要提取元素内的文本,您必须为 visibility_of_element_located 引入 WebDriverWait() 并且您可以使用以下任一解决方案:

The desired element is a dynamic element so to extract the text within the element you have to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following solutions:

  • 使用CSS_SELECTOR:

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "p.value.noWrap[data-bind$='MarketValue']"))).get_attribute("innerHTML"))

  • 使用XPATH:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//p[@class='value noWrap' and contains(@data-bind,'MarketValue')]"))).get_attribute("innerHTML"))
    

  • 注意:您必须添加以下导入:

  • Note : You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    

  • 这篇关于如何使用 Selenium - Python 检索 WebElement 的文本的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

    查看全文
    登录 关闭
    扫码关注1秒登录
    发送“验证码”获取 | 15天全站免登陆