将所有 celery 任务的日志消息发送到单个文件 [英] Send log messages from all celery tasks to a single file

问题描述

我想知道如何设置更具体的日志系统.我所有的任务都使用

I'm wondering how to setup a more specific logging system. All my tasks use

logger = logging.getLogger(__name__)

作为模块范围的记录器.

as a module-wide logger.

我希望 celery 将日志记录到celeryd.log"并将我的任务记录到tasks.log"，但我不知道如何让它工作.使用 django-celery 中的 CELERYD_LOG_FILE 我可以将所有与 celeryd 相关的日志消息路由到 celeryd.log，但没有在我的任务中创建的日志消息的踪迹.

I want celery to log to "celeryd.log" and my tasks to "tasks.log" but I got no idea how to get this working. Using CELERYD_LOG_FILE from django-celery I can route all celeryd related log messages to celeryd.log but there is no trace of the log messages created in my tasks.

推荐答案

注意:从 Celery 3.0 开始，此答案已过时，您现在使用 get_task_logger() 设置您的每任务记录器.请参阅Celery 3.0 新特性文档的日志记录部分 了解详情.

Note: This answer is outdated as of Celery 3.0, where you now use get_task_logger() to get your per-task logger set up. Please see the Logging section of the What's new in Celery 3.0 document for details.

Celery 对每个任务的日志记录都有专门的支持.请参阅有关该主题的任务文档:

Celery has dedicated support for logging, per task. See the Task documentation on the subject:

您可以使用工作日志记录器将诊断输出添加到工作日志:

You can use the workers logger to add diagnostic output to the worker log:

@celery.task()
def add(x, y):
    logger = add.get_logger()
    logger.info("Adding %s + %s" % (x, y))
    return x + y

有几个可用的日志级别，工作人员日志级别设置决定是否将它们写入日志文件.

There are several logging levels available, and the workers loglevel setting decides whether or not they will be written to the log file.

当然，您也可以简单地使用 print，因为写入标准输出/-err 的任何内容都将是也写入日志文件.

Of course, you can also simply use print as anything written to standard out/-err will be written to the log file as well.

在幕后，这仍然是标准的 Python 日志记录模块.您可以设置 CELERYD_HIJACK_ROOT_LOGGER option 为 False 以允许您自己的日志记录设置工作，否则 Celery 将为您配置处理.

Under the hood this is all still the standard python logging module. You can set the CELERYD_HIJACK_ROOT_LOGGER option to False to allow your own logging setup to work, otherwise Celery will configure the handling for you.

但是，对于任务，.get_logger() 调用确实允许您为每个单独的任务设置单独的日志文件.只需传入一个 logfile 参数，它就会将日志消息路由到那个单独的文件:

However, for tasks, the .get_logger() call does allow you to set up a separate log file per individual task. Simply pass in a logfile argument and it'll route log messages to that separate file:

@celery.task()
def add(x, y):
    logger = add.get_logger(logfile='tasks.log')
    logger.info("Adding %s + %s" % (x, y))
    return x + y 

最后但并非最不重要的一点是，您可以在 python 日志记录模块 并给它一个它自己的文件处理程序.我会使用 celery.signals.after_setup_task_logger 信号来设置它；在这里，我假设您的所有模块都位于一个名为 foo.tasks 的包中(如在 foo.tasks.email 和 foo.tasks.scaling 中):

Last but not least, you can just configure your top-level package in the python logging module and give it a file handler of it's own. I'd set this up using the celery.signals.after_setup_task_logger signal; here I assume all your modules live in a package called foo.tasks (as in foo.tasks.email and foo.tasks.scaling):

from celery.signals import after_setup_task_logger
import logging

def foo_tasks_setup_logging(**kw):
    logger = logging.getLogger('foo.tasks')
    if not logger.handlers:
        handler = logging.FileHandler('tasks.log')
        formatter = logging.Formatter(logging.BASIC_FORMAT) # you may want to customize this.
        handler.setFormatter(formatter)
        logger.addHandler(handler)
        logger.propagate = False

after_setup_task_logger.connect(foo_tasks_setup_logging)

现在任何名称以 foo.tasks 开头的记录器都将把它的所有消息发送到 tasks.log 而不是根记录器(它看不到任何这些消息中的一部分，因为 .propagate 是 False).

Now any logger whose name starts with foo.tasks will have all it's messages sent to tasks.log instead of to the root logger (which doesn't see any of these messages because .propagate is False).

这篇关于将所有 celery 任务的日志消息发送到单个文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！