如何从项目管道访问scrapy设置 [英] How to access scrapy settings from item Pipeline

问题描述

如何从项目管道访问 settings.py 中的 scrapy 设置.文档提到它可以通过扩展中的爬虫访问，但我没有看到如何访问管道中的爬虫.

How do I access the scrapy settings in settings.py from the item pipeline. The documentation mentions it can be accessed through the crawler in extensions, but I don't see how to access the crawler in the pipelines.

推荐答案

更新 (2021-05-04)
请注意，这个答案现在已经有大约 7 年的历史了，因此无法再确保其有效性.另外它使用的是Python2

UPDATE (2021-05-04)
Please note that this answer is now ~7 years old, so it's validity can no longer be ensured. In addition it is using Python2

从 your_spider.py 中访问 Scrapy 设置(在 settings.py 中定义)的方法很简单.所有其他答案都太复杂了.造成这种情况的原因是 Scrapy 文档的维护非常糟糕，再加上最近的许多更新&变化.在设置"中都没有文档 "如何访问设置"，也不在 设置 API"中 他们有没有费心提供任何可行的例子.这是一个示例，如何获取您当前的 USER_AGENT 字符串.

The way to access your Scrapy settings (as defined in settings.py) from within your_spider.py is simple. All other answers are way too complicated. The reason for this is the very poor maintenance of the Scrapy documentation, combined with many recent updates & changes. Neither in the "Settings" documentation "How to access settings", nor in the "Settings API" have they bothered giving any workable example. Here's an example, how to get your current USER_AGENT string.

只需将以下几行添加到 your_spider.py:

Just add the following lines to your_spider.py:

# To get your settings from (settings.py):
from scrapy.utils.project import get_project_settings
...
class YourSpider(BaseSpider):
    ...
    def parse(self, response):
        ...
        settings = get_project_settings()
        print "Your USER_AGENT is:
%s" % (settings.get('USER_AGENT'))
        ...

如您所见，无需使用 @classmethod 或重新定义 from_crawler() 或 __init__() 函数.希望这会有所帮助.

As you can see, there's no need to use @classmethod or re-define the from_crawler() or __init__() functions. Hope this helps.

PS.我仍然不确定为什么使用 from scrapy.settings import Settings 不能以同样的方式工作，因为它是更明显的选择导入?

PS. I'm still not sure why using from scrapy.settings import Settings doesn't work the same way, since it would be the more obvious choice of import?

这篇关于如何从项目管道访问scrapy设置的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！