与另一侧的连接以非清洁方式丢失 [英] Connection to other side was lost in a non-clean fashion

问题描述

from scrapy.spider import BaseSpider类 dmozSpider(BaseSpider):名称 = "dmoz"allowed_domains = ["dmoz.org"]start_urls = ["http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"]定义解析(自我，响应):文件名 = response.url.split("/")[-2]打开(文件名，'wb').写(response.body)

然后我运行scrapy crawl dmoz"然后我得到了这个错误:

<块引用>

2013-09-14 13:20:56+0700 [dmoz] 调试:重试 http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/>(失败 1 次):与另一端的连接在非清洁中丢失时尚.

有人知道如何解决这个问题吗?

解决方案

您需要检查您的互联网连接，或者如果您使用的是代理，请设置您的环境变量以进行代理身份验证.

在 Windows 中，请尝试以下步骤:

1. Win+R 键入systempropertiesadvanced"(不带引用)
2. 点击环境变量..."按钮
3. 添加 2 个新变量(用户/系统变量都可以):

<前>姓名 |价值------------+--------------------------------HTTP_PROXY |http://用户名:密码@主机:端口HTTPS_PROXY |https://用户名:密码@主机:端口

替代方式:设置代理环境

from scrapy.spider import BaseSpider

class dmozSpider(BaseSpider):
    name = "dmoz"
    allowed_domains = ["dmoz.org"]
    start_urls = [
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
    ]

    def parse(self, response):
        filename = response.url.split("/")[-2]
        open(filename, 'wb').write(response.body)

then I run "scrapy crawl dmoz" then I got this error:

2013-09-14 13:20:56+0700 [dmoz] DEBUG: Retrying http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/> (failed 1 times): Connection to other side was lost in a non-clean fashion.

Does anyone know how to fix this?

解决方案

You need to check your internet connection or if you're using proxy, set your environment variables for proxy authentication.

In windows, try these steps:

1. Win+R type 'systempropertiesadvanced' (without quote)
2. Click "Environment Variables..." button
3. Add 2 new variables (either user/system variable is fine):

name        | value
------------+--------------------------------  
HTTP_PROXY  | http://username:password@host:port 
HTTPS_PROXY | https://username:password@host:port

alternative way: setting-proxy-env

这篇关于与另一侧的连接以非清洁方式丢失的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！