子进程命令的实时输出 [英] live output from subprocess command

问题描述

我使用 python 脚本作为流体动力学代码的驱动程序.当需要运行模拟时，我使用 subprocess.Popen 来运行代码，将 stdout 和 stderr 的输出收集到一个 subprocess.PIPE --- 然后我可以打印(并保存到日志文件)输出信息，并检查是否有任何错误.问题是，我不知道代码的进展情况.如果我直接从命令行运行它，它会给我关于它在什么迭代、什么时间、下一个时间步是什么等的输出.

I'm using a python script as a driver for a hydrodynamics code. When it comes time to run the simulation, I use subprocess.Popen to run the code, collect the output from stdout and stderr into a subprocess.PIPE --- then I can print (and save to a log-file) the output information, and check for any errors. The problem is, I have no idea how the code is progressing. If I run it directly from the command line, it gives me output about what iteration its at, what time, what the next time-step is, etc.

有没有办法既存储输出(用于日志记录和错误检查)，又生成实时流输出?

我的代码的相关部分:

ret_val = subprocess.Popen( run_command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True )
output, errors = ret_val.communicate()
log_file.write(output)
print output
if( ret_val.returncode ):
    print "RUN failed

%s

" % (errors)
    success = False

if( errors ): log_file.write("

%s

" % errors)

最初我是通过 tee 管道 run_command 以便副本直接进入日志文件，并且流仍然直接输出到终端 - 但那我无法存储任何错误的方式(据我所知).

Originally I was piping the run_command through tee so that a copy went directly to the log-file, and the stream still output directly to the terminal -- but that way I can't store any errors (to my knowlege).

我目前的临时解决方案:

My temporary solution so far:

ret_val = subprocess.Popen( run_command, stdout=log_file, stderr=subprocess.PIPE, shell=True )
while not ret_val.poll():
    log_file.flush()

然后，在另一个终端中，运行 tail -f log.txt (s.t. log_file = 'log.txt').

then, in another terminal, run tail -f log.txt (s.t. log_file = 'log.txt').

推荐答案

TLDR for Python 3:

TLDR for Python 3:

import subprocess
import sys
with open('test.log', 'wb') as f: 
    process = subprocess.Popen(your_command, stdout=subprocess.PIPE)
    for c in iter(lambda: process.stdout.read(1), b''): 
        sys.stdout.buffer.write(c)
        f.buffer.write(c)

---

您有两种方法可以做到这一点，一种是从 read 或 readline 函数创建迭代器，然后执行:

---

You have two ways of doing this, either by creating an iterator from the read or readline functions and do:

import subprocess
import sys
with open('test.log', 'w') as f:  # replace 'w' with 'wb' for Python 3
    process = subprocess.Popen(your_command, stdout=subprocess.PIPE)
    for c in iter(lambda: process.stdout.read(1), ''):  # replace '' with b'' for Python 3
        sys.stdout.write(c)
        f.write(c)

或

import subprocess
import sys
with open('test.log', 'w') as f:  # replace 'w' with 'wb' for Python 3
    process = subprocess.Popen(your_command, stdout=subprocess.PIPE)
    for line in iter(process.stdout.readline, ''):  # replace '' with b'' for Python 3
        sys.stdout.write(line)
        f.write(line)

或者您可以创建一个 reader 和一个 writer 文件.将 writer 传递给 Popen 并从 reader

Or you can create a reader and a writer file. Pass the writer to the Popen and read from the reader

import io
import time
import subprocess
import sys

filename = 'test.log'
with io.open(filename, 'wb') as writer, io.open(filename, 'rb', 1) as reader:
    process = subprocess.Popen(command, stdout=writer)
    while process.poll() is None:
        sys.stdout.write(reader.read())
        time.sleep(0.5)
    # Read the remaining
    sys.stdout.write(reader.read())

通过这种方式，您将在 test.log 以及标准输出中写入数据.

This way you will have the data written in the test.log as well as on the standard output.

文件方法的唯一优点是您的代码不会阻塞.因此，您可以在此期间做任何您想做的事情，并以非阻塞方式随时从 reader 阅读.当您使用 PIPE 时，read 和 readline 函数将阻塞，直到分别向管道写入一个字符或向管道写入一行.

The only advantage of the file approach is that your code doesn't block. So you can do whatever you want in the meantime and read whenever you want from the reader in a non-blocking way. When you use PIPE, read and readline functions will block until either one character is written to the pipe or a line is written to the pipe respectively.

这篇关于子进程命令的实时输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！