如何在不失真的情况下打印和显示子进程 stdout 和 stderr 输出? [英] How can I print and display subprocess stdout and stderr output without distortion?

问题描述

也许外面有人可以帮助我解决这个问题.(我在 SO 上看到了许多与此类似的问题，但没有一个同时处理标准输出和标准错误，也没有处理像我这样的情况，因此是这个新问题.)

Maybe there's someone out in the ether that can help me with this one. (I have seen a number of similar questions to this on SO, but none deal with both standard out and standard error or deal with a situation quite like mine, hence this new question.)

我有一个 python 函数，它打开一个子进程，等待它完成，然后输出返回码，以及标准输出和标准错误管道的内容.在进程运行时，我还想显示两个管道在填充时的输出.我第一次尝试的结果是这样的:

I have a python function that opens a subprocess, waits for it to complete, then outputs the return code, as well as the contents of the standard out and standard error pipes. While the process is running, I'd like to also display the output of both pipes as they are populated. My first attempt has resulted in something like this:

process = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

stdout = str()
stderr = str()
returnCode = None
while True:
    # collect return code and pipe info
    stdoutPiece = process.stdout.read()
    stdout = stdout + stdoutPiece
    stderrPiece = process.stderr.read()
    stderr = stderr + stderrPiece
    returnCode = process.poll()

    # check for the end of pipes and return code
    if stdoutPiece == '' and stderrPiece == '' and returnCode != None:
        return returnCode, stdout, stderr

    if stdoutPiece != '': print(stdoutPiece)
    if stderrPiece != '': print(stderrPiece)

不过这有几个问题.因为 read() 一直读取到 EOF，所以 while 循环的第一行在子进程关闭管道之前不会返回.

There's a couple problems with this though. Because read() reads until an EOF, the first line of the while loop will not return until the subprocess closes the pipe.

我可以替换 read() 以支持 read(int) 但打印输出失真，在读取字符的末尾被切断.我可以 readline() 作为替代品，但是当有许多同时发生时，打印输出会因交替的输出行和错误而失真.

I could replace the read() in favor of read(int) but the printed output is distorted, cut off at the end of the read characters. I could readline() as a replacement, but the printed output is distorted with alternating lines of output and errors when there are many of both that occur at the same time.

也许有一个我不知道的 read-until-end-of-buffer() 变体?或者也许可以实现?

Perhaps there's a read-until-end-of-buffer() variant that I'm not aware of? Or maybe it can be implemented?

也许最好按照此 回答另一个帖子?但是，我只想在此函数中使用包装器.

Maybe it's best to implement a sys.stdout wrapper as suggested in this answer to another post? I would only want to use the wrapper in this function, however.

来自社区的任何其他想法?

Any other ideas from the community?

感谢您的帮助！:)

编辑:解决方案确实应该是跨平台的，但如果您有跨平台的想法，请分享它们以保持头脑风暴的进行.

EDIT: The solution really should be cross-platform, but if you have ideas that aren't, please share them away to keep the brainstorming going.

对于另一个我的 python 子进程头疼，看看我的另一个关于考虑子进程时间开销的问题一>.

For another one of my python subprocess head scratchers, take a look at another of my questions on accounting for subprocess overhead in timing.

推荐答案

使用 fcntl.fcntl，并使用 select.select 等待数据在任一管道中可用.例如:

Make the pipes non-blocking by using fcntl.fcntl, and use select.select to wait for data to become available in either pipe. For example:

# Helper function to add the O_NONBLOCK flag to a file descriptor
def make_async(fd):
    fcntl.fcntl(fd, fcntl.F_SETFL, fcntl.fcntl(fd, fcntl.F_GETFL) | os.O_NONBLOCK)

# Helper function to read some data from a file descriptor, ignoring EAGAIN errors
def read_async(fd):
    try:
        return fd.read()
    except IOError, e:
        if e.errno != errno.EAGAIN:
            raise e
        else:
            return ''

process = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
make_async(process.stdout)
make_async(process.stderr)

stdout = str()
stderr = str()
returnCode = None

while True:
    # Wait for data to become available 
    select.select([process.stdout, process.stderr], [], [])

    # Try reading some data from each
    stdoutPiece = read_async(process.stdout)
    stderrPiece = read_async(process.stderr)

    if stdoutPiece:
        print stdoutPiece,
    if stderrPiece:
        print stderrPiece,

    stdout += stdoutPiece
    stderr += stderrPiece
    returnCode = process.poll()

    if returnCode != None:
        return (returnCode, stdout, stderr)

请注意，fcntl 仅在类 Unix 平台上可用，包括 Cygwin.

Note that fcntl is only available on Unix-like platforms, including Cygwin.

如果您需要它在没有 Cygwin 的情况下在 Windows 上运行，这是可行的，但要困难得多.你必须:

If you need it to work on Windows without Cygwin, it's doable, but it's much, much tougher. You'll have to:

• 使用 pywin32 库调用本机 Win32 API
• 使用SetNamedPipeHandleState 和 PIPE_NOWAIT 使 stdout 和 stderr 管道无阻塞
• 使用WaitForMultipleObjects 而不是 select 等待数据变得可用
• 使用ReadFile 读取数据

• Use the pywin32 library to call through to the native Win32 API
• Use SetNamedPipeHandleState with PIPE_NOWAIT to make the stdout and stderr pipes non-blocking
• Use WaitForMultipleObjects instead of select to wait for data to become available
• Use ReadFile to read the data

这篇关于如何在不失真的情况下打印和显示子进程 stdout 和 stderr 输出?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！