Selenium 无法使用python 抓取Shopee 电子商务网站 [英] Selenium can not scrape Shopee e-commerce site using python
问题描述
我无法在 Shopee(一个电子商务网站)上提取产品的价格.
我查看了@dmitrybelyakov 解决的问题(链接:Scraping AJAX e-使用python的商业网站).
I am not able to pull the price of products on Shopee (a e-commercial site).
I have taken a look at the problem solved by @dmitrybelyakov (link: Scraping AJAX e-commerce site using python) .
该解决方案帮助我获得了产品的名称"和historical_sold",但我无法获得产品的价格.我在 Json 字符串中找不到价格值.因此,我尝试使用selenium 用xpath 拉取数据,但似乎失败了.
That solution helped me to get the 'name' of product and the 'historical_sold' but I can not get the price of the product. I can not find the price value in the Json string. Therefore, I tried to use selenium to pull data with xpath but it appeared to be failed.
电商网站链接:https://shopee.com.my/search?关键字=h370m
我的代码:
import time
from selenium import webdriver
import pandas as pd
path = r'C:\Users\\admin\\Desktop\\chromedriver_win32\\Chromedriver'
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('headless')
chrome_options.add_argument('window-size=1200x600')
browserdriver = webdriver.Chrome(executable_path = path,options=chrome_options)
link='https://shopee.com.my/search?keyword=h370m'
browserdriver.get(link)
productprice='//*[@id="main"]/div/div[2]/div[2]/div/div/div/div[2]/div/div/div[2]/div[1]/div/a/div/div[2]/div[1]'
productprice_printout=browserdriver.find_element_by_xpath(productname).text
print(productprice_printout)
当我运行该代码时,它显示如下错误通知:
When I run that code, it showed the error notification like this:
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="main"]/div/div[2]/div[2]/div/div/div/div[2]/div/div/div[2]/div[1]/div/a/div/div[2]/div[1]"}
请帮我了解 Shopee 上的产品价格!
Please help me to get the price of product on Shopee!
推荐答案
在 Shopee 使用 Selenium 和Python 您可以使用以下解决方案:
To extract the price of products on Shopee using Selenium and Python you can use the following solution:
代码块:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('start-maximized')
options.add_argument('disable-infobars')
options.add_argument('--disable-extensions')
browserdriver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
browserdriver.get('https://shopee.com.my/search?keyword=h370m')
WebDriverWait(browserdriver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[@class='shopee-modal__container']//button[text()='English']"))).click()
print([my_element.text for my_element in WebDriverWait(browserdriver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//span[text()='RM']//following::span[1]")))])
print("Program Ended")
控制台输出:
Console Output:
['430.00', '385.00', '435.00', '409.00', '479.00', '439.00', '479.00', '439.00', '439.00', '403.20', '369.00', '420.00', '479.00', '465.00', '465.00']
Program Ended
这篇关于Selenium 无法使用python 抓取Shopee 电子商务网站的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!