Scrapy: AttributeError: 'list' 对象没有属性 'iteritems' [英] Scrapy: AttributeError: 'list' object has no attribute 'iteritems'
问题描述
这是我关于堆栈溢出的第一个问题.最近我想使用linked-in-scraper,所以我下载并指示scrapy crawl linkedin.com"并得到以下错误信息.供您参考,我使用 anaconda 2.3.0 和 python 2.7.11.所有相关的包,包括scrapy和6个,在执行程序之前都通过pip更新.
This is my first question on stack overflow. Recently I want to use linked-in-scraper, so I downloaded and instruct "scrapy crawl linkedin.com" and get the below error message. For your information, I use anaconda 2.3.0 and python 2.7.11. All the related packages, including scrapy and six, are updated by pip before executing the program.
Traceback (most recent call last):
File "/Users/byeongsuyu/anaconda/bin/scrapy", line 11, in <module>
sys.exit(execute())
File "/Users/byeongsuyu/anaconda/lib/python2.7/site-packages/scrapy/cmdline.py", line 108, in execute
settings = get_project_settings()
File "/Users/byeongsuyu/anaconda/lib/python2.7/site-packages/scrapy/utils/project.py", line 60, in get_project_settings
settings.setmodule(settings_module_path, priority='project')
File "/Users/byeongsuyu/anaconda/lib/python2.7/site-packages/scrapy/settings/__init__.py", line 285, in setmodule
self.set(key, getattr(module, key), priority)
File "/Users/byeongsuyu/anaconda/lib/python2.7/site-packages/scrapy/settings/__init__.py", line 260, in set
self.attributes[name].set(value, priority)
File "/Users/byeongsuyu/anaconda/lib/python2.7/site-packages/scrapy/settings/__init__.py", line 55, in set
value = BaseSettings(value, priority=priority)
File "/Users/byeongsuyu/anaconda/lib/python2.7/site-packages/scrapy/settings/__init__.py", line 91, in __init__
self.update(values, priority)
File "/Users/byeongsuyu/anaconda/lib/python2.7/site-packages/scrapy/settings/__init__.py", line 317, in update
for name, value in six.iteritems(values):
File "/Users/byeongsuyu/anaconda/lib/python2.7/site-packages/six.py", line 599, in iteritems
return d.iteritems(**kw)
AttributeError: 'list' object has no attribute 'iteritems'
我知道这个错误源于 d 不是字典类型而是列表类型.由于错误来自scrapy上的代码,可能是scrapy包或六包的问题.我该如何尝试修复此错误?
I understand that this error stems from d is not the dictionary type but list type. And since the error is from the code on scrapy, maybe it is problem on scrapy package or six package. How can I try to fix this error?
这是来自scrapy.cfg的代码
This is code from scrapy.cfg
# Automatically created by: scrapy start project
#
# For more information about the [deploy] section see:
# http://doc.scrapy.org/topics/scrapyd.html
[settings]
default = linkedIn.settings
[deploy]
#url = http://localhost:6800/
project = linkedIn
推荐答案
这是由链接爬虫的 设置:
ITEM_PIPELINES = ['linkedIn.pipelines.LinkedinPipeline']
然而,ITEM_PIPELINES
应该是一个字典,根据文档:
However, ITEM_PIPELINES
is supposed to be a dict, according to the doc:
要激活 Item Pipeline 组件,您必须将其类添加到 ITEM_PIPELINES
设置中,如下例所示:
To activate an Item Pipeline component you must add its class to the
ITEM_PIPELINES
setting, like in the following example:
ITEM_PIPELINES = {
'myproject.pipelines.PricePipeline': 300,
'myproject.pipelines.JsonWriterPipeline': 800,
}
您在此设置中分配给类的整数值决定了它们运行的顺序:项目从价值较低的类到价值较高的类.通常将这些数字定义在 0-1000 范围内.
The integer values you assign to classes in this setting determine the order in which they run: items go through from lower valued to higher valued classes. It’s customary to define these numbers in the 0-1000 range.
根据这个问题,它曾经是一个列表,这就解释了为什么这个scraper使用列表.因此,您必须要求抓取工具的开发人员更新他们的代码,或者自己设置 ITEM_PIPELINES
.
According to this question, it used to be a list, which explains why this scraper uses a list.
So you will have to either ask your the developer of the scraper to update their code, or to set ITEM_PIPELINES
yourself.
这篇关于Scrapy: AttributeError: 'list' 对象没有属性 'iteritems'的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!