如何使用Selenium检索WebElement的文本-Python [英] How to retrieve the text of a WebElement using Selenium - Python

问题描述

我是Python和Web爬虫的新手，所以请多多包涵.我一直在尝试构建一个Web抓取工具来打开网页，登录并检索某个值.到目前为止，我已经能够打开网页并登录.但是，我根本无法找到一种方法来检索(打印)所需的值.这是我当前的代码:

I am new to Python and Web Scraping so please bear with me. I have been trying to build a web scraping tool to open a web page, log-in, and retrieve a certain value. Thus far, I have been able to open the web page and log-in. However, I simply cannot find a way to retrieve (print) the value that I require. This is what my current code looks like:

from selenium import webdriver
from bs4 import BeautifulSoup

driver = webdriver.Chrome(executable_path=r'C:/Users/User/Downloads/chromedriver.exe')

url = "xxxxxxxx"
driver.get(url)
driver.find_element_by_name("username").send_keys("xxxxx")
driver.find_element_by_name("password").send_keys("xxxxx")
elem = driver.find_element_by_css_selector("form#frmMain > a:nth-child(4)")
elem.click()

html = '''<p class="value noWrap" data-bind="text: MarketValue">R 4 516 469.32</p>'''
soup = BeautifulSoup(html, 'lxml')

for p in soup.find_all('p'):
    print(p.string)

driver.quit()

我需要的值嵌入在"R 4 516 469.32"上方的html变量中.但是，该值每天都会变化.我已经尝试过使用xpath和CSS，但是由于某些奇怪的原因，问题值似乎被隐藏了.我该如何动态引用该元素，以便每天都能检索到新值?

The value I require is embedded in the html variable above "R 4 516 469.32". However, this value changes on a daily basis. I have tried using xpath and css, but the value in question seems to be hidden for some odd reason. How can I refer to the element dynamically in order to be able to retrieve the new value every day?

请注意:由于该网址是用于公司目的的网站，因此我将该网址留空了.

Please note: I have blanked out the url as this is a website used for company purposes.

请帮助！

非常感谢

推荐答案

所需元素是动态元素，因此要提取元素中的文本，您必须为 visibility_of_element_located引入 WebDriverWait ()，您可以使用以下任一解决方案:

The desired element is a dynamic element so to extract the text within the element you have to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following solutions:

• 使用 CSS_SELECTOR :

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "p.value.noWrap[data-bind$='MarketValue']"))).get_attribute("innerHTML"))

使用 XPATH :

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//p[@class='value noWrap' and contains(@data-bind,'MarketValue')]"))).get_attribute("innerHTML"))

注意:您必须添加以下导入:

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

这篇关于如何使用Selenium检索WebElement的文本-Python的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！