无法运行 Scrapy 程序 [英] Can't run Scrapy program

问题描述

我一直在通过以下链接学习如何使用 Scrapy:

I have been learning how to work with Scrapy from the following link :

http://doc.scrapy.org/en/master/intro/教程.html

当我尝试运行在 Crawling(scrapy crawl dmoz) 部分中编写的代码时，出现以下错误:

When i try to run the code written in the Crawling(scrapy crawl dmoz) section, i get the following error:

AttributeError: 'module' 对象没有属性 'Spider'

但是，我将Spider"更改为spider"，但出现了一个新错误:

However, i changed "Spider" to "spider" and i got nothing but a new error:

TypeError: Error when calling the metaclass bases
module.__init__() takes at most 2 arguments (3 given)

我很困惑，有什么问题?任何帮助将不胜感激.谢谢.顺便说一下，我使用的是 Windows.

I'm so confused, what is the problem? Any help would highly be appreciated. Thanks. By the way, i am using Windows.

编辑(添加来源):

首先，我使用 Scrapy 创建了一个项目，方法是转到一个目录并通过 cmd 运行以下命令，如下所示:

First i created a project using Scrapy by going to a directory and running the following commands by cmd like so :

cd #DIRECTORY PATH#

scrapy startproject tutorial

这将在给定目录中创建一个名为 tutorial 的文件夹.教程文件夹包括:

This will create a folder named tutorial in the given directory. The tutorial folder consists :

教程/配置文件教程/初始化.py项目.py管道.py设置.py蜘蛛/初始化.py...

tutorial/ scrapy.cfg tutorial/ init.py items.py pipelines.py settings.py spiders/ init.py ...

然后我定义了我的项目:

Then i defined my items :

import scrapy

class DmozItem(scrapy.Item):
    title = scrapy.Field()
    link = scrapy.Field()
    desc = scrapy.Field()

之后，我创建了蜘蛛:

导入scrapy

class DmozSpider(scrapy.Spider):
    name = "dmoz"
    allowed_domains = ["dmoz.org"]
    start_urls = [
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
    ]

    def parse(self, response):
        filename = response.url.split("/")[-2]
        with open(filename, 'wb') as f:
            f.write(response.body)

然后，运行代码时，显示错误.我使用的是 Windows 7 64 位和 Python 2.7 32 位.

And after, when running the code,the error is shown. I am using windows 7 64 bit along with Python 2.7 32 bit.

编辑 2:

我尝试卸载并安装另一个 Scrapy 版本，但没有用.这是日志:

I tried uninstalling and installing another Scrapy version but it didn't work. Here is the log:

C:\Users\Novin Pendar\Desktop\FS\tutorial>scrapy crawl dmoz
2015-03-26 17:48:29+0430 [scrapy] INFO: Scrapy 0.16.5 started (bot: tutorial)
2015-03-26 17:48:29+0430 [scrapy] DEBUG: Enabled extensions: LogStats, TelnetCon
sole, CloseSpider, WebService, CoreStats, SpiderState
C:\Python27\lib\site-packages\scrapy-0.16.5-py2.7.egg\scrapy\__init__.pyc
Traceback (most recent call last):
  File "C:\Python27\lib\runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "C:\Python27\lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Python27\lib\site-packages\scrapy-0.16.5-py2.7.egg\scrapy\cmdline.py"
, line 156, in <module>
    execute()
  File "C:\Python27\lib\site-packages\scrapy-0.16.5-py2.7.egg\scrapy\cmdline.py"
, line 131, in execute
    _run_print_help(parser, _run_command, cmd, args, opts)
  File "C:\Python27\lib\site-packages\scrapy-0.16.5-py2.7.egg\scrapy\cmdline.py"
, line 76, in _run_print_help
    func(*a, **kw)
  File "C:\Python27\lib\site-packages\scrapy-0.16.5-py2.7.egg\scrapy\cmdline.py"
, line 138, in _run_command
    cmd.run(args, opts)
  File "C:\Python27\lib\site-packages\scrapy-0.16.5-py2.7.egg\scrapy\commands\cr
awl.py", line 43, in run
    spider = self.crawler.spiders.create(spname, **opts.spargs)
  File "C:\Python27\lib\site-packages\scrapy-0.16.5-py2.7.egg\scrapy\command.py"
, line 33, in crawler
    self._crawler.configure()
  File "C:\Python27\lib\site-packages\scrapy-0.16.5-py2.7.egg\scrapy\crawler.py"
, line 40, in configure
    self.spiders = spman_cls.from_crawler(self)
  File "C:\Python27\lib\site-packages\scrapy-0.16.5-py2.7.egg\scrapy\spidermanag
er.py", line 35, in from_crawler
    sm = cls.from_settings(crawler.settings)
  File "C:\Python27\lib\site-packages\scrapy-0.16.5-py2.7.egg\scrapy\spidermanag
er.py", line 31, in from_settings
    return cls(settings.getlist('SPIDER_MODULES'))
  File "C:\Python27\lib\site-packages\scrapy-0.16.5-py2.7.egg\scrapy\spidermanag
er.py", line 22, in __init__
    for module in walk_modules(name):
  File "C:\Python27\lib\site-packages\scrapy-0.16.5-py2.7.egg\scrapy\utils\misc.
py", line 65, in walk_modules
    submod = __import__(fullpath, {}, {}, [''])
  File "tutorial\spiders\dmoz_spider.py", line 3, in <module>
    class DmozSpider(scrapy.Spider):
AttributeError: 'module' object has no attribute 'Spider'

编辑 3:

问题解决了.我下载了最新版本的 Scrapy(0.24) 并安装.一切都很好.只是想对和我以前有同样问题的人说，这样，他们会节省很多时间.谢谢.

The problem is solved. I downloaded the latest version of Scrapy(0.24) and installed. Everything worked so great. Just wanted to say to people who have the same problem that i used to, so, they will save them a lot of time. Thanks.

推荐答案

如果你的安装正确.试试这个

If your installation is correct.Try this

检查工作文件夹中的任何scrapy.py或scrapy.pyc.如果存在，重命名它.不要将Spider更改为<代码>蜘蛛

Check any scrapy.py or scrapy.pyc, in the working folder.If exists, rename it.Dont change Spider to spider

这篇关于无法运行 Scrapy 程序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！