Scrapy Spider 无法正确抓取 [英] Scrapy Spider not scraping correctly

问题描述

我在 Windows Vista 上使用 Python.org 2.7 64 位 shell.我安装了 Scrapy，它似乎很稳定并且可以工作.但是，我复制了以下简单的代码:

I am using Python.org 2.7 64 bit shell on Windows Vista. I have Scrapy installed and it seems to be stable and working. However, I have copied the following simple piece of code:

from scrapy.spider import BaseSpider
from scrapy.selector import HtmlXPathSelector

class MySpider(BaseSpider):
        name = "craig"
        allowed_domains = ["craigslist.org"]
        start_urls = ["http://sfbay.craigslist.org/sfc/npo/"]

        def parse(self, response):
            hxs = HtmlXPathSelector(response)
            titles = hxs.select("//p")
            for titles in titles:
                title = titles.select("a/text()").xpath()
                link = titles.select("a/@href").xpath()
                print title, link

包含在此 Youtube 视频中:

Contained in this Youtube video:

http://www.youtube.com/watch?v=1EFnX1UkXVU当我运行此代码时，我收到警告:

http://www.youtube.com/watch?v=1EFnX1UkXVU When I run this code I get the warning:

    hxs = HtmlXPathSelector(response)
C:\Python27\mrscrap\mrscrap\spiders\test.py:11: ScrapyDeprecationWarning: Call to deprecated function select. Use .xpath() instead.
  titles = hxs.select("//p")
c:\Python27\lib\site-packages\scrapy\selector\unified.py:106: ScrapyDeprecationWarning: scrapy.selector.HtmlXPathSelector is deprecated, ins
.Selector instead.
  for x in result]
C:\Python27\mrscrap\mrscrap\spiders\test.py:13: ScrapyDeprecationWarning: Call to deprecated function select. Use .xpath() instead.
  title = titles.select("a/text()").extract()
C:\Python27\mrscrap\mrscrap\spiders\test.py:14: ScrapyDeprecationWarning: Call to deprecated function select. Use .xpath() instead.
  link = titles.select("a/@href").extract()

Scrapy 的一些语法最近是否发生了变化，以至于 .extract() 不再有效?我曾尝试使用 .xpath()，但这会引发错误，指出 .xpath() 需要两个参数，但我不确定在那里使用什么.

Has some of the syntax of Scrapy changed recently so that .extract() is no longer valid? I've tried using .xpath() instead, but this throws up an error saying that two arguments are required for .xpath(), but I'm not sure what to use there.

有什么想法吗?

谢谢

推荐答案

参考其他答案，应该是

title = titles.xpath("a/text()").extract()

这篇关于Scrapy Spider 无法正确抓取的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！