在python中访问子进程的标准输出 [英] Access standard output of a sub process in python

问题描述

在将子进程发送到主进程之前，如何访问它们的标准输出?我正在使用multiprocessing.Pool模块来生成子进程池.

How can I access the stdout of child processes prior to sending them to main process? I am using multiprocessing.Pool module to generate child process pools.

推荐答案

主进程和子进程都共享相同的标准输入和标准输出文件描述符.他们无法控制其他人写给他们的东西.您唯一可以做的就是用主进程可以控制的其他内容替换子代中的stdin和stdout.例如，您可以将虚拟文件对象子类化，例如 StringIO ，然后通过 Queue :

The main process and the children all share the same standard input and standard output file descriptors. They have no control over what the other writes to them. The only thing you can do is replace stdin and stdout in the children with something else that the main process can control. As an example you could subclass a dummy file object like StringIO and redirect the data that the children write to this object to the parent via a Queue:

import sys
from multiprocessing import Queue, Pool, current_process
from StringIO import StringIO

class MyStringIO(StringIO):
    def __init__(self, queue, *args, **kwargs):
        StringIO.__init__(self, *args, **kwargs)
        self.queue = queue
    def flush(self):
        self.queue.put((current_process().name, self.getvalue()))
        self.truncate(0)

def initializer(queue):
     sys.stderr = sys.stdout = MyStringIO(queue)

def task(num):
     print num
     sys.stdout.flush()
     return num ** 2

q = Queue()
pool = Pool(3, initializer, [q])

for _ in pool.map(task, range(5)):
    proc, out = q.get()
    print proc, "got", out

这应该打印出这样的内容:

This should print something like this:

PoolWorker-1 got 0
PoolWorker-1 got 3
PoolWorker-1 got 4
PoolWorker-2 got 1
PoolWorker-3 got 2

别忘了在task末尾调用sys.{stdout,stderr}.flush()，否则，不会有任何内容写入队列.

Don't forget to call sys.{stdout,stderr}.flush() at the end of task otherwise, nothing will be written to the queue.

这篇关于在python中访问子进程的标准输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！