使用硒在一页上单击多个项目 [英] Clicking multiple items on one page using selenium

问题描述

我的主要目的是去这个特定的网站，单击每个产品，有足够的时间从被单击的产品中抓取数据，然后回去单击页面中的另一个产品，直到所有产品都被点击为止.并抓取(我未包含的抓取代码).

My main purpose is to go to this specific website, to click each of the products, have enough time to scrape the data from the clicked product, then go back to click another product from the page until all the products are clicked through and scraped (The scraping code I have not included).

我的代码打开chrome重定向到我想要的网站，生成了一个按class_name单击的链接列表.这是我要坚持的部分，我相信我需要一个for循环来迭代链接列表以单击并返回到原始链接.但是，我不知道为什么这行不通.

My code opens up chrome to redirect to my desired website, generates a list of links to click by class_name. This is the part I am stuck on, I would believe I need a for-loop to iterate through the list of links to click and go back to the original. But, I can't figure out why this won't work.

这是我的代码:

import csv
import time
from selenium import webdriver
import selenium.webdriver.chrome.service as service
import requests
from bs4 import BeautifulSoup


url = "https://www.vatainc.com/infusion/adult-infusion.html?limit=all"
service = service.Service('path to chromedriver')
service.start()
capabilities = {'chrome.binary': 'path to chrome'}
driver = webdriver.Remote(service.service_url, capabilities)
driver.get(url)
time.sleep(2)
links = driver.find_elements_by_class_name('product-name')


for link in links:
    link.click()
    driver.back()
    link.click()

推荐答案

对于您的问题，我还有另一种解决方法.

I have another solution to your problem.

当我测试您的代码时，它表现出奇怪的行为.修复了我使用xpath时遇到的所有问题.

When I tested your code it showed a strange behaviour. Fixed all problems that I had using xpath.

url = "https://www.vatainc.com/infusion/adult-infusion.html?limit=all"
driver.get(url)
links = [x.get_attribute('href') for x in driver.find_elements_by_xpath("//*[contains(@class, 'product-name')]/a")]
htmls = []
for link in links:
    driver.get(link)
    htmls.append(driver.page_source)

我没有来回移动而是保存了所有链接(称为链接)并遍历此列表.

Instead of going back and forward I saved all links (named as links) and iterate over this list.

这篇关于使用硒在一页上单击多个项目的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！