诱导WebDriverWait获得特定元素 [英] Inducing WebDriverWait for specific elements

问题描述

此问题是我之前的问题的后续跟进内容(通过< div>在硒中).我正在努力从grailed.com(

This question is a follow up to my previous question (Inconsistency in scraping through <div>'s in Selenium). I'm working on scraping all of the Air Jordan Data off of grailed.com (https://www.grailed.com/designers/jordan-brand/hi-top-sneakers). I am storing the size, model, url, and image url in an object. I currently have a program that scrolls through the entire feed and fetches all of this. Everything works except finding the image url. The image URL seems to require inducing an explicit wait, which @KunduK suggested. I'm trying to implement his solution so that I can pull each image in my for loop:

 while True and len(sneakers) < sneaker_count:
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    # Get sneakers currently on page and add to sneakers list
    feed = driver.find_elements_by_class_name('feed-item')
    images = WebDriverWait(driver, 10).until(
      EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".feed-item .listing-cover-photo>img")))
    for item in feed:
      ...

当前，代码一次获取一组图像.我正在尝试获取供稿中的商品"期间的图片堵塞.我想要类似images = WebDriverWait(driver, 10).until(EC.visibility_of_elements_located((By.SOME SELECTOR", ???)))的东西，但我真的不知道如何使用'item'元素找到这些东西.有人可以帮忙吗?

Currently the code fetches the image in a group at once. I am trying to fetch the image during the "for item in feed" block. I want something like images = WebDriverWait(driver, 10).until(EC.visibility_of_elements_located((By.SOME SELECTOR", ???))) but I really don't know how to find these using the 'item' element. Can anyone help?

推荐答案

使用 python ，您必须为该应用引入 WebDriverWait visibility_of_all_elements_located()，您可以使用以下任一定位器策略:

To scrape the image url from each image using Selenium and python you have to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

• 使用CSS_SELECTOR:

driver.get('https://www.grailed.com/designers/jordan-brand/hi-top-sneakers')
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a.product-card-container")))])

使用XPATH:

driver.get('https://www.grailed.com/designers/jordan-brand/hi-top-sneakers')
print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//a[@class='product-card-container']")))])

控制台输出:

Console Output:

['https://www.grailed.com/products/57773-jordan-brand-air-jordan-1-retro-high-og-court-purple', 'https://www.grailed.com/products/57803-jordan-brand-air-jordan-1-retro-high-og-obsidian', 'https://www.grailed.com/products/57759-jordan-brand-air-jordan-1-retro-high-og-2017-royal', 'https://www.grailed.com/products/57760-jordan-brand-air-jordan-1-retro-high-og-2018-shadow', 'https://www.grailed.com/products/59036-jordan-brand-air-jordan-4-retro-og-2019-bred', 'https://www.grailed.com/products/115772-jordan-brand-jordan-1-retro-high-og-pine-green', 'https://www.grailed.com/products/57817-jordan-brand-air-jordan-1-retro-high-og-shattered-backboard-3-0', 'https://www.grailed.com/products/61668-jordan-brand-travis-scott-travis-scott-x-air-jordan-4-retro-cactus-jack', 'https://www.grailed.com/products/114979-jordan-brand-air-jordan-1-retro-high-og-unc-to-chi', 'https://www.grailed.com/products/97122-jordan-brand-air-jordan-1-retro-high-og-fearless', 'https://www.grailed.com/products/97133-jordan-brand-air-jordan-11-bred-2019', 'https://www.grailed.com/products/61725-jordan-brand-air-jordan-4-retro-cool-grey', 'https://www.grailed.com/products/57762-jordan-brand-air-jordan-1-retro-high-og-banned-2016-banned-bred', 'https://www.grailed.com/products/87098-jordan-brand-travis-scott-travis-scott-x-air-jordan-6-retro-olive', 'https://www.grailed.com/products/57768-jordan-brand-air-jordan-1-retro-high-og-bred-toe', 'https://www.grailed.com/products/112831-jordan-brand-air-jordan-1-retro-high-og-royal-toe', 'https://www.grailed.com/products/111383-jordan-brand-air-jordan-4-retro-black-cat-2020', 'https://www.grailed.com/products/58136-jordan-brand-travis-scott-travis-scott-x-air-jordan-1-retro-high-og-mocha', 'https://www.grailed.com/products/57825-jordan-brand-air-jordan-1-retro-high-og-turbo-green', 'https://www.grailed.com/products/111377-jordan-brand-off-white-air-jordan-5-retro-sp-muslin']

注意:您必须添加以下导入:

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

这篇关于诱导WebDriverWait获得特定元素的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！