在scrapy教程中扭曲了严重的未处理错误 [英] twisted critical unhandled error on scrapy tutorial

问题描述

我是编程新手，我正在尝试使用scrapy教程学习scrapy:http://doc.scrapy.org/en/latest/intro/tutorial.html

I'm new in programming and I'm trying to learn scrapy, using scrapy tutorial: http://doc.scrapy.org/en/latest/intro/tutorial.html

所以我运行了scrapy crawl dmoz"命令并得到了这个错误:

So I ran "scrapy crawl dmoz" command and got this error:

2015-07-14 16:11:02 [scrapy] INFO: Scrapy 1.0.1 started (bot: tutorial)
2015-07-14 16:11:02 [scrapy] INFO: Optional features available: ssl, http11
2015-07-14 16:11:02 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE':     'tu
torial.spiders', 'SPIDER_MODULES': ['tutorial.spiders'], 'BOT_NAME':   'tutorial'}

2015-07-14 16:11:05 [scrapy] INFO: Enabled extensions: CloseSpider, TelnetConsol
e, LogStats, CoreStats, SpiderState
Unhandled error in Deferred:
2015-07-14 16:11:06 [twisted] CRITICAL: Unhandled error in Deferred:
2015-07-14 16:11:07 [twisted] CRITICAL:

我使用的是 windows 7 和 python 2.7.有谁知道是什么问题?我该如何解决?

I'm using windows 7 and python 2.7. Anybody knows what's the problem? How could I fix that?

我的蜘蛛文件代码是:

# This package will contain the spiders of your Scrapy project
#
# Please refer to the documentation for information on how to create and manage
# your spiders.
import scrapy


class DmozSpider(scrapy.Spider):
    name = "dmoz"
    allowed_domains = ["dmoz.org"]
     start_urls = [
    "http://www.dmoz.org/computers/programming/languages/python/books/",
    "http://www.dmoz.org/computer/programming/languages/python/resources/"
]

    def parse(self, response):
        filename = response.url.split("/")[-2] + '.html'
        with open(filename,'wb') as f:
            f.write(response.body)

items.py 代码:

items.py code:

import scrapy

class DmozItem(scrapy.Item):
    title = scrapy.Field()
    link = scrapy.Field()
    desc = scrapy.Field()

点列表:

• bootstrap-admin (0.3.3)
• cffi (1.1.2)
• 特性 (14.3.0)
• 密码学 (0.9.3)
• cssselect (0.9.1)
• Django (1.7.7)
• django-auth-ldap (1.2.4)
• django-debug-toolbar (1.3.0)
• django-mssql (1.6.2)
• django-pyodbc (0.2.6)
• django-pyodbc-azure (1.2.2)
• django-redator (0.2.3)
• django-reversion (1.8.5)
• django-summernote (0.6.0)
• django-windows-tools (0.1.1)
• django-wysiwyg-redactor (0.4.3.2)
• enum34 (1.0.4)
• ez-setup (0.9)
• flup (1.0.2)
• idna (2.0)
• ipaddress (1.0.13)
• iso8601 (0.1.4)
• 日志记录 (0.4.9.6)
• lxml (3.4.4)
• 机械化 (0.2.5)
• MySQL-python (1.2.4)
• pbr (0.10.8)
• 枕头 (2.7.0)
• pip (7.1.0)
• pyasn1 (0.1.8)
• pyasn1-modules (0.0.6)
• pycparser (2.14)
• pymongo (2.6)
• pyodbc (3.0.7)
• pyOpenSSL (0.15.1)
• pypm (1.4.3)
• python-ldap (2.4.18)
• pythonselect (1.3)
• pywin32 (218.3)
• queuelib (1.2.2)
• Scrapy (1.0.1)
• 硒 (2.44.0)
• 服务标识 (14.0.0)
• 设置工具 (18.0.1)
• 六 (1.9.0)
• sqlparse (0.1.15)
• 装卸工 (1.3.0)
• 扭曲 (15.2.1)
• virtualenv (1.11.6)
• virtualenv-clone (0.2.5)
• virtualenvwrapper (4.3.2)
• virtualenvwrapper-powershell (12.7.8)
• w3lib (1.11.0)
• xlrd (0.9.2)
• zope.interface (4.1.2)

感谢大家对我糟糕的英语的关注和同情，这不是我的母语.

Thx for the attention and sry for my poor English, isn't my native language.

推荐答案

我也开始学习scrapy，遇到和你一样的问题.折腾了一个下午，终于发现是pywin32模块只下载不安装造成的.您可以尝试在 cmd 中输入以下命令以完成 pywin32 模块安装并再次尝试抓取:

I'm beginning to learn scrapy as well and encounter the same question with yours. After struggling with it for an afternoon, finally I found it's due to the pywin32 module only download without install. You can try input the command below in the cmd to finish the pywin32 module install and try crawl again:

python python27\scripts\pywin32_postinstall.py -install

python python27\scripts\pywin32_postinstall.py -install

希望能帮到你！

这篇关于在scrapy教程中扭曲了严重的未处理错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！