subprocess命令的实时输出 [英] live output from subprocess command

问题描述

我使用python脚本作为流体力学代码的驱动程序。当运行模拟时，我使用 subprocess.Popen 运行代码，将stdout和stderr的输出收集到一个子进程中.PIPE ---然后我可以打印（并保存到日志文件）输出信息，并检查是否有任何错误。问题是，我不知道代码如何进行。如果我直接从命令行运行它，它会输出关于它在什么时候的迭代，什么时候，下一个时间步骤等等的输出。

有没有办法存储输出（用于记录和错误检查），并且还会产生实时流输出？

相关部分我的代码：

  ret_val = subprocess.Popen（run_command，stdout = subprocess.PIPE，stderr = subprocess.PIPE，shell = True ）
输出，errors = ret_val.communicate（）
 log_file.write（输出）
打印输出
 if（ret_val.returncode）：
打印运行失败\\ n\\\
％s \\\
\\\
％（errors）
 success = False 
 
 if（errors）：log_file.write（\\\
\\\
％ s \\\
\\\
％errors）
  

最初我是管道 run_command 通过 tee ，以便一个副本直接转到日志文件，流仍然直接输出到终端 - 但是这样我可以'

---

编辑：

临时解决方案：

  ret_val = subprocess.Popen（run_command，stdout = log_file，stderr = subprocess.PIPE，shell =真的）
而不是ret_val.poll（）：
 log_file.flush（）
  

然后在另一个终端中运行 tail -f log.txt （st

解决方案

你有两个通过从读取或 readline 函数创建一个迭代器，并执行以下操作：

  import subprocess 
 import sys 
 with open（'test.log'，'w'）as f：
 process = subprocess.Popen（your_command，stdout = subprocess.PIPE）
 for c中的iter（lambda：process.stdout.read（1），''）：
 sys.stdout.write（c ）
 f.write（c）
  

或

  import subprocess 
 import sys 
 with open（'test.log'，'w'）as f：
 process = subprocess .Popen（your_command，stdout = subprocess.PIPE）
 for iter（process.stdout.readline，''）
 sys.stdout.write（line）
 f.write（线条）
  

或者您可以创建一个阅读器和作者文件。将作者传递给 Popen 并从阅读器读取

  import io 
 import time 
 import subprocess 
 import sys 
 
 filename ='test.log'
 with io.open（filename，'wb'）as writer，io.open（filename，'rb'，1）as reader：
 process = subprocess。 Popen（command，stdout = writer）
 while process.poll（）是None：
 sys.stdout.write（reader.read（））
 time.sleep（0.5）
＃读取剩余的
 sys.stdout.write（reader.read（））
  

这样您就可以将数据写入 test.log 以及标准输出。

文件方法的唯一优点是您的代码不会阻止。所以你可以在任何时候做任何你想要的东西，每当你想从阅读器以非阻塞的方式阅读。当您使用 PIPE ，读取和 readline 函数将阻止直到将一个字符写入管道，或者将一行写入管道。

I'm using a python script as a driver for a hydrodynamics code. When it comes time to run the simulation, I use subprocess.Popen to run the code, collect the output from stdout and stderr into a subprocess.PIPE --- then I can print (and save to a log-file) the output information, and check for any errors. The problem is, I have no idea how the code is progressing. If I run it directly from the command line, it gives me output about what iteration its at, what time, what the next time-step is, etc.

Is there a way to both store the output (for logging and error checking), and also produce a live-streaming output?

The relevant section of my code:

ret_val = subprocess.Popen( run_command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True )
output, errors = ret_val.communicate()
log_file.write(output)
print output
if( ret_val.returncode ):
    print "RUN failed\n\n%s\n\n" % (errors)
    success = False

if( errors ): log_file.write("\n\n%s\n\n" % errors)

Originally I was piping the run_command through tee so that a copy went directly to the log-file, and the stream still output directly to the terminal -- but that way I can't store any errors (to my knowlege).

---

Edit:

Temporary solution:

ret_val = subprocess.Popen( run_command, stdout=log_file, stderr=subprocess.PIPE, shell=True )
while not ret_val.poll():
    log_file.flush()

then, in another terminal, run tail -f log.txt (s.t. log_file = 'log.txt').

解决方案

You have two ways of doing this, either by creating an iterator from the read or readline functions and do:

import subprocess
import sys
with open('test.log', 'w') as f:
    process = subprocess.Popen(your_command, stdout=subprocess.PIPE)
    for c in iter(lambda: process.stdout.read(1), ''):
        sys.stdout.write(c)
        f.write(c)

or

import subprocess
import sys
with open('test.log', 'w') as f:
    process = subprocess.Popen(your_command, stdout=subprocess.PIPE)
    for line in iter(process.stdout.readline, ''):
        sys.stdout.write(line)
        f.write(line)

Or you can create a reader and a writer file. Pass the writer to the Popen and read from the reader

import io
import time
import subprocess
import sys

filename = 'test.log'
with io.open(filename, 'wb') as writer, io.open(filename, 'rb', 1) as reader:
    process = subprocess.Popen(command, stdout=writer)
    while process.poll() is None:
        sys.stdout.write(reader.read())
        time.sleep(0.5)
    # Read the remaining
    sys.stdout.write(reader.read())

This way you will have the data written in the test.log as well as on the standard output.

The only advantage of the file approach is that your code doesn't block. So you can do whatever you want in the meantime and read whenever you want from the reader in a non-blocking way. When you use PIPE, read and readline functions will block until either one character is written to the pipe or a line is written to the pipe respectively.

这篇关于subprocess命令的实时输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！