Celery记录器配置 [英] Celery logger configuration

问题描述

我正在使用Django 1.10，python 3.5和celery 4.1.0 我正在尝试将芹菜任务信息记录到文件中.因此，我按照芹菜文档中的建议进行了尝试-

I'm using Django 1.10, python 3.5 and celery 4.1.0 I'm trying to log celery tasks info into a file. So I tried as suggested in celery documentation -

from celery.utils.log import get_task_logger

logger = get_task_logger(__name__)

并尝试在任务内记录一条消息-

and tried to log a message inside the task -

logger.info(message)

我希望它可以登录到我的默认记录器.但事实并非如此.因此，我在设置中添加了一个名为"celery.task"的专用记录器(据我从文档中了解):

I expected it to log to my default logger. But it didn't. So I added to settings a dedicated logger named 'celery.task' (as I understand from documentation):

LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
    'require_debug_false': {
        '()': 'django.utils.log.RequireDebugFalse',
    },
    'require_debug_true': {
        '()': 'django.utils.log.RequireDebugTrue',
    },
    'require_test_false': {
        '()': 'myapp.utils.classes.logging.RequireTestFalse',
    },
    'suppress_deprecated': {
        '()': 'myapp.utils.classes.logging.SuppressDeprecated'
    }
},
'handlers': {
    'console': {
        'level': 'INFO',
        'class': 'logging.StreamHandler',
        'formatter': 'json',
        'filters': ['suppress_deprecated']
    },
    'celery_file': {
        'level': 'INFO',
        'class': 'myapp.utils.classes.logging.SBRotatingFileHandler',
        'maxBytes': 1024 * 1024 * 200,  # 200 MB
        'backupCount': 10,
        'formatter': 'json',
        'filename': BASE_DIR + '/../log/celery.log',
    }
},
'loggers': {
    'django': {
        'handlers': ['console', 'file'],
        'level': LOG_LEVEL,
        'propagate': True,
    },
    'celery.task': {
        'handlers': ['console', 'celery_file'],
        'level': 'INFO',
        'propagate': True,
            },
}

但是我仍然没有在celery.log文件或默认日志文件中看到celery任务的日志.

But I still don't see logs from celery task not in the celery.log file nor in the default log file.

仅当以'-f'启动celery worker时-它将日志写入该文件 有什么想法吗?

Only when starting celery worker with '-f' - it writes logs to that file Any ideas?

我正在尝试使用'after_setup_task_logger'更新celery.task logger处理程序，以成功使用我的logging.config(在设置中)存在的处理程序. 我尝试了以下方法:

I'm trying to use 'after_setup_task_logger' to update the celery.task logger handler to use a handler that exists in my logging.config (in settings) unsuccessfully. I've tried the following:

 @celery.signals.after_setup_task_logger.connect
 def after_setup_logging(logger, **kwargs):
     logging_settings = settings.LOGGING
     celery_handler = logging_settings['handlers']['celery_file']
     logger.addHandler(celery_handler)

但这不起作用.我正在

AttributeError: 'dict' object has no attribute 'createLock'

这意味着未正确创建处理程序. 因此，我尝试从记录"对象中获取处理程序.但是在logging._handlers和logging._handlersList

Which means the handler was not properly created. So I tried getting the handler from 'logging' object. But I don't see my handler in both logging._handlers and logging._handlersList

更新: 那才是最终对我有用的东西:

UPDATE: That's what finally worked for me:

def create_celery_logger_handler(logger, propagate):
    # 209715200 is 1024 * 1024 * 200 or 200 MB, same as in settings
    celery_handler = RotatingFileHandler(
        settings.CELERY_LOG_FILE,
        maxBytes=209715200,
        backupCount=10
    )
    celery_formatter = jsonlogger.JsonFormatter(settings.LOGGING['formatters']['json']['format'])
    celery_handler.setFormatter(celery_formatter)

    logger.addHandler(celery_handler)
    logger.logLevel = settings.LOG_LEVEL
    logger.propagate = propagate


@celery.signals.after_setup_task_logger.connect
def after_setup_celery_task_logger(logger, **kwargs):
    """ This function sets the 'celery.task' logger handler and formatter """
    create_celery_logger_handler(logger, True)


@celery.signals.after_setup_logger.connect
def after_setup_celery_logger(logger, **kwargs):
    """ This function sets the 'celery' logger handler and formatter """
    create_celery_logger_handler(logger, False)

推荐答案

对于它的价值，这是我将celery配置为使用Django日志记录设置的方式:

For what it's worth, this is how I configured celery to use my Django logging settings:

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
from celery.signals import setup_logging

# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'app.settings')

app = Celery('app')

# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')

@setup_logging.connect
def config_loggers(*args, **kwags):
    from logging.config import dictConfig
    from django.conf import settings
    dictConfig(settings.LOGGING)

# Load task modules from all registered Django app configs.
app.autodiscover_tasks()

就是这样-更改之后，我的Django日志记录设置适用于芹菜，包括我已设置为将日志消息发送到闲置状态的日志记录.

That was about it - after that change my django logging settings worked for celery, including the logging I had setup to send log messages over to slack.

这篇关于Celery记录器配置的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！