python& Scrapy:Scrapy版本的问题 [英] Python & Scrapy: Issue with Scrapy version

问题描述

我正在使用 Ubuntu 14.04 x64 ，我已经按照 Scrapy 文档使用 pip 安装软件包:

I'm using Ubuntu 14.04 x64, I've followed the Scrapy docs to install the package using pip:

pip install scrapy

然后，我跟随示例项目的初始化并尝试执行示例蜘蛛:

Then I followed the initialization of the example project and try to execute the example spider:

scrapy crawl example

我得到这个错误:

2015-02-23 10:23:42+0100 [scrapy] INFO: Scrapy 0.14.4 started (bot: example)
2015-02-23 10:23:42+0100 [scrapy] DEBUG: Enabled extensions: LogStats, TelnetConsole, CloseSpider, WebService, CoreStats, MemoryUsage, SpiderState
Traceback (most recent call last):
  File "/usr/local/bin/scrapy", line 11, in <module>
    sys.exit(execute())
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 132, in execute
    _run_print_help(parser, _run_command, cmd, args, opts)
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 97, in _run_print_help
    func(*a, **kw)
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 139, in _run_command
    cmd.run(args, opts)
  File "/usr/lib/python2.7/dist-packages/scrapy/commands/crawl.py", line 43, in run
    spider = self.crawler.spiders.create(spname, **opts.spargs)
  File "/usr/lib/python2.7/dist-packages/scrapy/command.py", line 34, in crawler
    self._crawler.configure()
  File "/usr/lib/python2.7/dist-packages/scrapy/crawler.py", line 36, in configure
    self.spiders = spman_cls.from_crawler(self)
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 37, in from_crawler
    return cls.from_settings(crawler.settings)
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 33, in from_settings
    return cls(settings.getlist('SPIDER_MODULES'))
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 23, in __init__
    for module in walk_modules(name):
  File "/usr/lib/python2.7/dist-packages/scrapy/utils/misc.py", line 65, in walk_modules
    submod = __import__(fullpath, {}, {}, [''])
  File "/home/alvaro/vagrantenvs/example/example/spiders/example_spider.py", line 3, in <module>
    class ExampleSpider(scrapy.Spider):
AttributeError: 'module' object has no attribute 'Spider'

我与pip freeze | grep 'Scrapy'进行了检查，以检查 Scrapy 的安装版本，并且得到了:

I checked with pip freeze | grep 'Scrapy' to check the installed version of Scrapy and I got:

Scrapy==0.24.4

但是正如您在回溯错误中看到的那样:

but as you can see in the traceback error:

2015-02-23 10:23:42+0100 [scrapy] INFO: Scrapy 0.14.4 started (bot: example)

1. 我尝试卸载并重新安装
2. 我也尝试过更新pip，卸载scrapy并重新安装
3. 我尝试使用 apt-get
安装它

1. I tried to uninstall and reinstall
2. I tried also to update pip, uninstall scrapy and reinstall
3. I tried to install it using apt-get

---

• 为什么如果我安装了最新版本的(0.24.4)，它似乎正在执行0.14.4?
• 此问题与Scrapy，Pip或两者都有关系吗?

---

• Why If I installed the latest version (0.24.4) it seems it's executing 0.14.4 ?
• Is this issue related with Scrapy, Pip or both ?

这是文件example_spider.py:

Here it is the file example_spider.py:

import scrapy

class ExampleSpider(scrapy.Spider):
    name = "example"
    allowed_domains = ["dmoz.org"]
    start_urls = [
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
    ]

    def parse(self, response):
        filename = response.url.split("/")[-2]
        with open(filename, 'wb') as f:
            f.write(response.body)

如果我按照@aberna的建议执行scrapy version，我会得到:

If I execute scrapy version as @aberna sugested I got:

Scrapy 0.14.4

推荐答案

那时我也遇到了类似的问题，这是因为 Ubuntu 程序包弄乱了 pip 包.

I had a similar issue back then and It was because the Ubuntu package was messing with the pip package.

当我使用 pip 安装Scrapy并尝试运行它时， scrapy 说该软件包缺少某些依赖项，然后我安装了 Scrapy 使用apt-get，在那里我搞砸了软件包的版本.

When I installed Scrapy using pip and try to run it, scrapy says that the package was missing some dependencies and then I installed Scrapy using apt-get and there is where I screwed up with the package version.

我建议您从 Ubuntu 和 Pip 中删除这两个软件包，然后仅使用pip重新安装:

I recommend you to remove both packages from Ubuntu and Pip and then reinstall only using pip:

apt-get remove --purge python-scrapy
pip uninstall scrapy

这些操作完成后，您可以使用pip重新安装，然后，如果您执行scrapy version，则应该以最新版本作为响应，对于您来说，最新版本应为 0.24.4 .

After those actions finished you can install it again using pip and then if you do scrapy version you should get as response the newest version, in your case it should be 0.24.4.

您可以在Scrapy文档中阅读以下内容:

You can read this in Scrapy Documentation:

python-scrapy是官方的debian存储库提供的另一种软件包，它已经过时了，Scrapy团队也不支持.

python-scrapy is a different package provided by official debian repositories, it’s very outdated and it isn’t supported by Scrapy team.

要在Ubuntu中安装Scrapy，他们会说:

To install Scrapy in Ubuntu they say:

1. 将用于将Scrapy软件包签名的GPG密钥导入APT密钥环:

1. Import the GPG key used to sign Scrapy packages into APT keyring:

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 627220E7

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 627220E7

使用以下命令创建/etc/apt/sources.list.d/scrapy.list文件 以下命令:

Create /etc/apt/sources.list.d/scrapy.list file using the following command:

echo'deb http://archive.scrapy.org/ubuntu scrapy main'| sudo tee/etc/apt/sources.list.d/scrapy.list

echo 'deb http://archive.scrapy.org/ubuntu scrapy main' | sudo tee /etc/apt/sources.list.d/scrapy.list

3.更新软件包列表并安装scrapy-0.24软件包:

3.Update package lists and install the scrapy-0.24 package:

sudo apt-get更新&& sudo apt-get install scrapy-0.24

sudo apt-get update && sudo apt-get install scrapy-0.24

在此处检查:在Ubuntu上Scrapy

这篇关于python& Scrapy:Scrapy版本的问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！