逐行读取子进程标准输出 [英] read subprocess stdout line by line

问题描述

我的python脚本使用子进程调用一个非常嘈杂的linux实用程序.我想将所有输出存储到日志文件中并向用户显示其中的一些.我认为以下方法可行，但在实用程序产生大量输出之前，输出不会显示在我的应用程序中.

#fake_utility.py，随着时间的推移生成大量输出导入时间我 = 0为真:打印十六进制(i)*512我 += 1时间.睡眠(0.5)#过滤器输出导入子流程proc = subprocess.Popen(['python','fake_utility.py'],stdout=subprocess.PIPE)对于 proc.stdout 中的行:#真正的代码在这里做过滤打印 "test:", line.rstrip()

我真正想要的行为是过滤器脚本打印从子进程接收到的每一行.有点像 tee 但使用 python代码.

我错过了什么?这甚至可能吗?

<小时>

更新:

如果将 sys.stdout.flush() 添加到 fake_utility.py，则代码在 python 3.1 中具有所需的行为.我正在使用 python 2.6.您会认为使用 proc.stdout.xreadlines() 与 py3k 的工作方式相同，但事实并非如此.

<小时>

更新 2:

这是最小的工作代码.

#fake_utility.py，随着时间的推移生成大量输出导入系统，时间对于范围内的我(10):打印我sys.stdout.flush()时间.睡眠(0.5)#逐行显示输出导入子流程proc = subprocess.Popen(['python','fake_utility.py'],stdout=subprocess.PIPE)#适用于python 3.0+#for proc.stdout 中的行:对于 iter(proc.stdout.readline,'') 中的行:打印 line.rstrip()

解决方案

我上次使用 Python 已经很久了，但我认为问题出在 proc.stdout 中的语句 for line，它在迭代之前读取整个输入.解决方案是使用 readline() 代替:

#filters 输出导入子流程proc = subprocess.Popen(['python','fake_utility.py'],stdout=subprocess.PIPE)为真:行 = proc.stdout.readline()如果不是行:休息#真正的代码在这里做过滤打印 "test:", line.rstrip()

当然你仍然需要处理子进程的缓冲.

注意:根据文档 带有迭代器的解决方案应该等同于使用 readline()，除了预读缓冲区，但是(或者正是因为这个)提议的更改确实对我产生了不同的结果(Windows XP 上的 Python 2.5).>

My python script uses subprocess to call a linux utility that is very noisy. I want to store all of the output to a log file and show some of it to the user. I thought the following would work, but the output doesn't show up in my application until the utility has produced a significant amount of output.

#fake_utility.py, just generates lots of output over time
import time
i = 0
while True:
   print hex(i)*512
   i += 1
   time.sleep(0.5)

#filters output
import subprocess
proc = subprocess.Popen(['python','fake_utility.py'],stdout=subprocess.PIPE)
for line in proc.stdout:
   #the real code does filtering here
   print "test:", line.rstrip()

The behavior I really want is for the filter script to print each line as it is received from the subprocess. Sorta like what tee does but with python code.

What am I missing? Is this even possible?

---

Update:

If a sys.stdout.flush() is added to fake_utility.py, the code has the desired behavior in python 3.1. I'm using python 2.6. You would think that using proc.stdout.xreadlines() would work the same as py3k, but it doesn't.

---

Update 2:

Here is the minimal working code.

#fake_utility.py, just generates lots of output over time
import sys, time
for i in range(10):
   print i
   sys.stdout.flush()
   time.sleep(0.5)

#display out put line by line
import subprocess
proc = subprocess.Popen(['python','fake_utility.py'],stdout=subprocess.PIPE)
#works in python 3.0+
#for line in proc.stdout:
for line in iter(proc.stdout.readline,''):
   print line.rstrip()

解决方案

It's been a long time since I last worked with Python, but I think the problem is with the statement for line in proc.stdout, which reads the entire input before iterating over it. The solution is to use readline() instead:

#filters output
import subprocess
proc = subprocess.Popen(['python','fake_utility.py'],stdout=subprocess.PIPE)
while True:
  line = proc.stdout.readline()
  if not line:
    break
  #the real code does filtering here
  print "test:", line.rstrip()

Of course you still have to deal with the subprocess' buffering.

Note: according to the documentation the solution with an iterator should be equivalent to using readline(), except for the read-ahead buffer, but (or exactly because of this) the proposed change did produce different results for me (Python 2.5 on Windows XP).

这篇关于逐行读取子进程标准输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！