使用 Python 子进程从标准输出上的 C 程序捕获输出?期待吗? [英] Capture output from C program on stdout using Python subprocess? Pexpect?

问题描述

我想从我这样启动的 C 程序中捕获输出:

I want to capture output from a C program I'm launching like this:

p = subprocess.Popen(["make", "run_pci"],
                     stdout=subprocess.PIPE,
                     cwd="/home/ecorbett/hello_world_pthread")
for ln in p.stdout:

唯一的问题是在 C 程序完成之前我不会得到输出，而实际上我需要在程序运行时逐行获取输出.更复杂的是，我必须解析每一行(我只需要行中的某些数据).

Only problem is I dont get output until the C program is done, when in fact I need to get output line by line as the program is running. And to further complicate matters, I have to parse each line ( i only need certain data from lines).

例如，这里是一些示例输出:(我需要捕获Tile on Tile #")

For example, here is some sample output: (I need to capture "Thread on Tile #")

blahblah blah Thread blahblah blah tile 1: On 
blahblah blah Thread blahblah blah tile 2: OFF 
blahblah blah Thread blahblah blah tile 3 : Disable

我注意到我在下面链接的文章似乎有同样的问题.我在想如何让它适应我的情况?

I noticed the article I linked below seems to have the same problem. I was trying to figure how to adapt it to my situation?

获取实时输出来自 ffmpeg 用于进度条(PyQt4，stdout)

Python 新手，非常感谢示例代码！！！

Python newbie, so example code is greatly appreciated!!!

推荐答案

你不能那样使用 p.stdout；如果您要求整个标准输出"，这将仅在进程终止(或管道缓冲区填充，这可能需要很长时间)时可用.

You can't use p.stdout like that; if you ask for "the whole stdout", this will only be available upon process termination (or pipe buffer filling, which could take a long time).

您需要逐行读取进程的标准输出.

You need to read from the process's stdout line by line.

while True:
    ln = p.stdout.readline()
    if '' == ln:
        break
    m = re.search("Thread (?P<id>\d+)", ln);
    if m:
        # use m.group() to extract information
        # e.g. m.group('id') will hold the 12345 from "Thread 12345"

最好将 stdout 设置为行缓冲(通常尽可能完全缓冲)，但我认为这只能在被调用的程序中完成.

It would also be best if stdout could be set to line buffered (usually it is fully buffered wherever possible), but I think this can only be done from within the called program.

这里有两个缓冲区需要考虑.一种是 C 程序的输出缓冲区.这可能不存在(无缓冲输出)、行缓冲或完全缓冲(1K、4K 或 8K 是一些可能的大小).

We have two buffers to consider here. One is the C program's output buffer. This may be nonexistent (unbuffered output), line buffered, or fully buffered (1K, 4K or 8K are some possible sizes).

在程序中，调用了printf()".输出如下:

Within the program, a "printf()" is called. The output goes:

• out，如果没有缓冲
• 进入缓冲区；然后输出缓冲区中所有以换行符结尾的行，如果行缓冲；
• 进入缓冲区；然后输出第一个 4K，如果用 4K 缓冲区完全缓冲并且缓冲区大于 4K.

现在输出进入 Python 的管道.这也可能是完全缓冲(stdout)或行缓冲(readline).所以输出是:

Now the output enters Python's pipe. This again may be fully buffered (stdout) or line buffered (readline). So the output goes:

• 对于 python 程序的逻辑，如果管道中有一个完整的换行符终止行并且我们正在使用 readline
• 到缓冲区，如果管道中少于 4K 并且我们使用for ln in stdout".

在最后一种情况下，缓冲区将以 4K 块的形式进入 Python 逻辑.

In this last case, the buffer will go in 4K chunks to the Python logic.

现在让我们想象一个缓冲行 C 程序每秒输出一行，长度为 1K 个字符，到 Python 程序(如果 C 程序是完全缓冲的，那么可以完成！)

Let us now imagine a line buffered C program outputting one line, 1K characters long, each second, to a Python program (if the C program is fully buffered, there's not very much that can be done!)

循环读取标准输出，我们会看到(在 for 循环内):

Reading stdout in cycle, we would see (inside the for loop):

• t = 0 ...没有
• t = 1 ... 什么都没有(缓冲区已满 50%)
• t = 2 ... 什么都没有(缓冲区已满 75%)
• t = 3 ... 四行输出
• t = 4 ...没有...

通读readline我们会得到:

Reading through readline we would get:

• t = 0 ...一行
• t = 1 ...一行
• t = 2 ...一行
• t = 3 ...一行

示例

在这里，我运行ping -c 3 -i 2 127.0.0.1"以便以两秒的间隔向本地主机发送三个数据包.一次 ping 运行大约需要 6 秒钟.我从 ping 读取输出，并打印时间戳.ping 的整个输出足够小，可以放入 Python 的全缓冲区.

Here I run "ping -c 3 -i 2 127.0.0.1" in order to get three packets to localhost at two seconds interval. One run of ping takes around six seconds. I read the output from ping, and print a timestamp. The whole output of ping is small enough that it fits in Python's full-buffer.

#!/usr/bin/python

import subprocess
from time import gmtime, strftime

p = subprocess.Popen(["ping", "-c", "3", "-i", "2", "127.0.0.1"],
                 stdout=subprocess.PIPE)

for ln in p.stdout:
    print strftime("%H:%M:%S", gmtime()) + " received " + ln

# Now I start the same process again, reading the input the other way.

p = subprocess.Popen(["ping", "-c", "3", "-i", "2", "127.0.0.1"],
                 stdout=subprocess.PIPE)

while True:
    ln = p.stdout.readline()
    if '' == ln:
            break
    print strftime("%H:%M:%S", gmtime()) + " received " + ln

我在 Linux 机器上收到的输出如预期的那样:

The output I receive on my Linux box is, as expected:

(nothing for the first six seconds)
15:40:10 received PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
15:40:10 received 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.037 ms
15:40:10 received 64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.034 ms
15:40:10 received 64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.031 ms
15:40:10 received
15:40:10 received --- 127.0.0.1 ping statistics ---
15:40:10 received 3 packets transmitted, 3 received, 0% packet loss, time 3998ms
15:40:10 received rtt min/avg/max/mdev = 0.031/0.034/0.037/0.002 ms

15:40:10 received PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
15:40:10 received 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.041 ms
15:40:12 received 64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.039 ms
15:40:14 received 64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.035 ms
15:40:14 received
15:40:14 received --- 127.0.0.1 ping statistics ---
15:40:14 received 3 packets transmitted, 3 received, 0% packet loss, time 3999ms
15:40:14 received rtt min/avg/max/mdev = 0.035/0.038/0.041/0.005 ms

这篇关于使用 Python 子进程从标准输出上的 C 程序捕获输出?期待吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！