未找到爬虫蜘蛛 [英] scrapy spider not found

问题描述

我正在尝试重现本次演讲的代码:

I'm trying to reproduce the code of this talk:

https://www.youtube.com/watch?v=eD8XVXLlUTE

当我尝试运行蜘蛛时:

scrapy crawl talkspider_basic

我收到此错误:

raise KeyError("Spider not found: {}".format(spider_name))
KeyError: 'Spider not found: talkspider_basic'

蜘蛛的代码是:

from scrapy.spiders import BaseSpider
from scrapy.selector import HtmlXPathSelector
from scrapy.contrib.linkextractors.sgml import  SgmlLinkExtractor
from scrapy.contrib.loader import XPathItemLoader
from pytexas.items import  PytexasItem

class TalkspiderBasicSpider(BaseSpider):
    name = "talkspider_basic"
    allowed_domains = ["www.pytexas.org"]
    start_urls = ['http://wwww.pytexas.org/2013/schedule']

    def parse(self, response):
        hxs = HtmlXPathSelector(response)
        dls = hcs.select('///dl')
        for dl in dls:
            times = dl.select('dt/text()').extract()
            titles = dl.select('dd/a/text()').extract()
            for time, title in zip(times,titles):
                title = title.strip()
                yield PytexasItem(title=title,time= time)

物品的代码是:

from scrapy.item import Item, Field

class PytexasItem(Item):
    title = Field()
    time = Field()
    speaker = Field()
    description = Field()

项目名称和蜘蛛文件的名称是

The name of the project and of the spider's file are

pytexas

和

talk_spider_basic.py

talk_spider_basic.py

分别，所以我认为不会因为名称而有任何冲突.

respectively, so I don't think that there is any conflict because of the names.

它具有默认结构:

pytexas/     
  scrapy.cfg    
  pytexas/    
    items.py   
    pipelines.py   
    settings.py   
    spiders/   
      __init__.py   
      talk_spider_basic.py    

推荐答案

根据 Github 问题 #2254.因为某些模块已被弃用.例如 scrapy.contrib.

According Github Issues #2254. Because some module is deprecated.Like scrapy.contrib.

所以你应该做出一些改变.

So you should make some change.

来自:

from scrapy.contrib.linkextractors.sgml import  SgmlLinkExtractor
from scrapy.contrib.loader import XPathItemLoader

致:

from scrapy.linkextractors import LinkExtractor
from scrapy.loader import XPathItemLoader

这篇关于未找到爬虫蜘蛛的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！