元素内部&LT;跨度&GT;标签找不到回来了美丽的汤 [英] Element inside <span> tag not found back with Beautiful Soup
本文介绍了元素内部&LT;跨度&GT;标签找不到回来了美丽的汤的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我已经用美丽的汤与Python从网站抓取数据的尝试。
I have tried using Beautiful Soup with Python to crawl data from a website.
当我检查网站本身我看到以下内容:
When I inspect the website itself I see the following:
"<span id="test"> 567 </span>"
但是当我用美丽的汤,我看到的是:
but when I use Beautiful soup, all I see is:
"<span id="test"></span>"
这是因为如果该号码已被隐藏的安全和保护的目的,但我怎样才能从这个分析的信息?我认为这是一个动态的JS元素,但我不知道肯定,以及如何访问它。
It is as if the number has been hidden for security and protection purposes but how can I get this info from parsing? I think it is a dynamic JS element but I don’t know for sure and how to access it.
推荐答案
下面是一个例子code实现它:
Here is an example code to implement it:
import scrapy
from selenium import webdriver
from bs4 import BeautifulSoup as bs
class SomeSpider(scrapy.Spider):
name = "some"
allowed_domains = ["yourdomain.com"]
def __init__(self, *a, **kw):
super(SomeSpider, self).__init__(*a, **kw)
self.start_urls = ['http://www.yoururl.com',]
firefox_profile = webdriver.FirefoxProfile()
self.driver = webdriver.Firefox(firefox_profile=firefox_profile)
def parse(self, response):
self.driver.get(response.url)
page = TextResponse(response.url, body=self.driver.page_source, encoding='utf-8')
#HERE whatever you want do do eg. text = page.xpath('//span[@id="test"]').extract_first()
这篇关于元素内部&LT;跨度&GT;标签找不到回来了美丽的汤的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文