登录 Scrapy [英] Logging in Scrapy

问题描述

我在登录scrapy时遇到问题，而且我能找到的大部分内容都已过时.

I am having trouble with logging in scrapy, and most of what I can find is out of date.

我已经在 settings.py 文件和文档中设置了 LOG_FILE="log.txt"，这应该可以工作:

I have set LOG_FILE="log.txt" in the settings.py file and from the documentation, this should work:

Scrapy 在每个 Spider 实例中提供了一个记录器，可以像这样访问和使用:

Scrapy provides a logger within each Spider instance, that can be accessed and used like this:

import scrapy

class MySpider(scrapy.Spider):

    name = 'myspider'
    start_urls = ['http://scrapinghub.com']

    def parse(self, response):
        self.logger.info('Parse function called on %s', response.url)

但是当我这样做时:

class MySpider(CrawlSpider):
    #other code
    def parse_page(self,response):
        self.logger.info("foobar")

我一无所获.如果我设置

I get nothing. If I set

logger = logging.basicConfig(filename="log.txt",level=logging.INFO)

在我的文件顶部，在我导入之后，它会创建一个日志文件，并且默认输出被记录得很好，但是

At the top of my file, after my imports, it creates a log file, and the default output gets logged just fine, but

class MySpider(CrawlSpider):
    #other code
    def parse_page(self,response):
        logger.info("foobar")

未能露面.我也试过把它放在 __init__ 类中，例如:

Fails to make an appearance. I have also tried putting it in the class __init__, as such:

def __init__(self, *a, **kw):
    super(FanfictionSpider, self).__init__(*a, **kw)
    logging.basicConfig(filename="log.txt",level=logging.INFO)

我再次没有输出到文件，只有控制台，并且 foobar 没有显示.有人可以指导我如何正确登录 Scrapy 吗?

I once again get no output to the file, just to the console, and foobar does not show up. Can someone please direct me on how to correctly log in Scrapy?

推荐答案

对于日志记录，我只是把它放在蜘蛛类上:

For logging I just put this on the spider class:

import logging
from scrapy.utils.log import configure_logging 


class SomeSpider(scrapy.Spider):
    configure_logging(install_root_handler=False)
    logging.basicConfig(
        filename='log.txt',
        format='%(levelname)s: %(message)s',
        level=logging.INFO
    )

这会将所有scrapy输出作为log.txt文件放入项目根目录

This will put all scrapy output into the project root directory as a log.txt file

如果你想手动记录一些东西，你不应该使用scrapy logger，它已被弃用.只用python一个

If you want to log something manually you shouldn't use the scrapy logger, it's deprecated. Just use the python one

import logging
logging.error("Some error")

这篇关于登录 Scrapy的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！