使用多处理的python日志记录，Windows中的root记录器有所不同 [英] python logging with multiprocessing, root logger different in windows

问题描述

我尝试使用多处理进行日志记录，发现在Windows下，我将在子进程中获得不同的root记录器，但是在Linux下可以.

I tried logging with multiprocessing, and found under windows, I will get different root logger in child process, but under Linux that is ok.

测试代码:

main.py:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import logging
import multiprocessing
from mymod import func

def m_func():
    server = multiprocessing.Process(target=func, args=())
    server.start()

logger = logging.getLogger()
#print 'in global main: ', logger

if __name__ == '__main__':
    print 'in main: ', logger
    m_func()

mymod.py:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import logging

logger = logging.getLogger()
# print 'in global func: ', logger

def func():
    print 'in func: ', logger

在Linux下，结果是:

Under Linux, the result is:

in main:  <logging.RootLogger object at 0x10e4d6d90>
in func:  <logging.RootLogger object at 0x10e4d6d90>

但是在Windows 7(64位)下，我将在main和func之间得到不同的root记录器:

But under Windows 7, 64 bit, I will get different root logger between main and func:

in main:  <logging.RootLogger object at 0x00000000021FFD68>
in func:  <logging.RootLogger object at 0x00000000023BC898>

如果我在主脚本中初始化root记录器，如何在Windows下将诸如子进程级别之类的设置保留在Windows下?

And If I initialize root logger in main scripts, how can I keep the settings such as level in child process under windows?

推荐答案

在我看来，这可以链接到

It seems to me that this could be linked to the following platform-dependant behaviour:

16.6.3.2.视窗 由于Windows缺少os.fork()，因此它具有一些额外的限制:

16.6.3.2. Windows Since Windows lacks os.fork() it has a few extra restrictions:

(...)

全局变量

请记住，如果代码在子进程中运行，则会尝试访问 全局变量，那么它看到的值(如果有)可能不相同 作为Process.start当时父流程中的值 叫.

Bear in mind that if code run in a child process tries to access a global variable, then the value it sees (if any) may not be the same as the value in the parent process at the time that Process.start was called.

但是，只是模块级常量的全局变量会导致 没问题.

However, global variables which are just module level constants cause no problems.

根据您的问题，我认为这会导致logging.basicConfig()调用未到达您的所有进程.一种解决方案是让您的子进程登录到Queue(使用QueueHandler)，并在您的主进程中有一个专用线程来监听队列.

From your question, I assume that this results in a logging.basicConfig() call that is not reaching all your processes. A solution to this is to have your child processes to log to a Queue (using QueueHandler), and have a dedicated thread in your main process that will listen to the queue.

这篇关于使用多处理的python日志记录，Windows中的root记录器有所不同的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！