Python BeautifulSoup返回空列表 [英] Python BeautifulSoup Returning Empty List

问题描述

我正在尝试创建一个Python脚本，以使用BeautifulSoup从tcgplayer.com获取Yugioh卡价格.当您在该网站上搜索卡时，它会返回一个搜索结果页面，其中包含来自不同卖家的几种价格.我的目标是拉低所有这些价格.在下面的示例中，我打开了名为"A"细胞育种设备的卡片的搜索​​结果:

I'm trying to create a Python script to pull prices of Yugioh Card prices from tcgplayer.com using BeautifulSoup. When you search for a card on this website, it returns a page of search results with several prices from different sellers. My goal is to pull all of these prices. In the below example, I'm opening the search result for a card called "A" Cell Breeding Device:

import urllib2
from bs4 import BeautifulSoup
html = urllib2.open('http://shop.tcgplayer.com/productcatalog/product/show?newSearch=false&ProductType=All&IsProductNameExact=false&ProductName=%22A%22%20Cell%20Breeding%20Device')
soup = BeautifulSoup(html, 'lxml')
soup.find_all('span', {'class': 'scActualPrice largetext pricegreen'})

几天前，正确运行soup.find_all行为我提供了我所需的信息.但是，现在运行它会给我一个空数组[].我已经搜索了很多有关BeautifulSoup返回一个空数组的内容，但是我不确定它们是否适用于我，因为它几天前就可以正常工作了.有人可以帮我指出正确的方向吗?预先谢谢你！

A few days ago, running the soup.find_all line correctly gave me the information I needed. However, running this now gives me an empty array []. I've searched pretty extensively about BeautifulSoup returning an empty array, but I'm not sure if any of them apply to me since it was working just fine a few days ago. Can someone help point me in the right direction? Thank you in advance!

推荐答案

您应使用硒使用真实的浏览器进行抓取:

You should use selenium to scrap using real browser:

from selenium import webdriver

driver = webdriver.Chrome('/path/to/chromedriver')
driver.get('http://shop.tcgplayer.com/productcatalog/product/show?newSearch=false&ProductType=All&IsProductNameExact=false&ProductName=%22A%22%20Cell%20Breeding%20Device')
prices = driver.find_elements_by_css_selector('.scActualPrice')
for element in prices:
    print(element.text)
driver.quit()

这篇关于Python BeautifulSoup返回空列表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！