空python进程挂在连接[sys.stderr.flush()]上 [英] Empty python process hangs on join [sys.stderr.flush()]

问题描述

Python专家我需要您的帮助.我遇到了非常奇怪的行为:空python进程挂起.看起来它派生了一些锁定的资源.

Python guru I need your help. I faced quite strange behavior: empty python Process hangs on joining. Looks like it forks some locked resource.

环境:

• Python版本:3.5.3
• 操作系统:Ubuntu 16.04.2 LTS
• 内核:4.4.0-75通用

问题描述:

1)我有一个带有线程的记录器，该线程可在后台处理消息并为此线程排队.记录器源代码(略有简化).

1) I have a logger with thread to handle messages in background and queue for this thread. Logger source code (a little bit simplified).

2)我有一个使用记录器的简单脚本(只需代码即可显示我的问题):

2) And I have a simple script which uses my logger (just code to display my problem):

import os
from multiprocessing import Process
from my_logging import get_logger


def func():
    pass


if __name__ == '__main__':

    logger = get_logger(__name__)
    logger.start()
    for _ in range(2):
        logger.info('message')

    proc = Process(target=func)
    proc.start()
    proc.join(timeout=3)
    print('TEST PROCESS JOINED: is_alive={0}'.format(proc.is_alive()))

    logger.stop()
    print('EXIT')

有时此测试脚本会挂起.脚本挂在加入过程"proc"上(脚本完成执行时).测试过程"proc"仍然有效.

Sometimes this test script hangs. Script hangs on joining process "proc" (when script completes execution). Test process "proc" stay alive.

要重现此问题，您可以循环运行脚本:

To reproduce this problem you can run the script in loop:

$ for i in {1..100} ; do /opt/python3.5.3/bin/python3.5 test.py ; done

调查:

Strace显示以下内容:

Strace shows following:

strace: Process 25273 attached
futex(0x2275550, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, ffffffff

然后我弄清楚了进程挂起的地方.它挂在冲洗STDERR的多处理模块文件process.py，第269行(python3.5.3)中:

And I figured out the place where process hangs. It hangs in multiprocessing module, file process.py, line 269 (python3.5.3), on flushing STDERR:

...
267    util.info('process exiting with exitcode %d' % exitcode)
268    sys.stdout.flush()
269    sys.stderr.flush()
...

如果第269行有注释，则脚本始终总是成功完成.

If line 269 commented the script completes successfully always.

我的想法:

默认记录.StreamHandler使用sys.stderr作为流.

By default logging.StreamHandler uses sys.stderr as stream.

如果记录器将数据刷新到STDERR时已分叉进程，则进程上下文将获得一些锁定的资源，并进一步挂起刷新STDERR.

If process has been forked when logger flushing data to STDERR, process context gets some locked resource and further hangs on flushing STDERR.

一些解决问题的方法:

• 使用python2.7.我无法用python2.7复制它.也许时间安排使我无法重现该问题.
• 使用进程在记录器中而不是线程中处理消息.

您对此行为有任何想法吗?问题出在哪儿?我在做错什么吗?

推荐答案

此行为似乎与此问题有关:

It looks like this behaviour is related to this issue: http://bugs.python.org/issue6721

这篇关于空python进程挂在连接[sys.stderr.flush()]上的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！