解析 pexpect 输出 [英] Parsing pexpect output

问题描述

我正在尝试实时解析程序块缓冲的输出，这意味着在进程结束之前输出不可用.我需要的只是逐行解析、过滤和管理输出中的数据，因为它可能会运行数小时.

I'm trying to parse in real time the output of a program block-buffered, which means that output is not available until the process ends. What I need is just to parse line by line, filter and manage data from the output, as it could run for hours.

我尝试使用 subprocess.Popen() 捕获输出，但是，正如您可能猜到的，Popen 无法管理这种行为，它一直在缓冲直到进程结束.

I've tried to capture the output with subprocess.Popen(), but yes, as you may guess, Popen can't manage this kind of behavior, it keeps buffering until end of process.

from subprocess import Popen, PIPE

p = Popen("my noisy stuff ", shell=True, stdout=PIPE, stderr=PIPE)
for line in p.stdout.readlines():
    #parsing text and getting data

所以我找到了 pexpect，它实时打印输出，因为它将标准输出视为文件，或者我什至可以做一个肮脏的技巧打印出一个文件并在函数之外解析它.但是好吧，它太脏了，即使对我来说也是如此 ;)

So I found pexpect, which prints the output in real time, as it treats the stdout as a file, or I could even do a dirty trick printing out a file and parsing it outside the function. But ok, it is too dirty, even for me ;)

import pexpect
import sys

pexpect.run("my noisy stuff", logfile=sys.stdout)

但我想这应该是一种更好的pythonic方式来做到这一点，只需像子进程一样管理标准输出.波本可以.我该怎么做?

But I guess it should a better pythonic way to do this, just manage the stdout like subprocess. Popen does. How can I do this?

运行 J.F. 提案:

Running J.F. proposal:

这是一次故意错误的审核，大约需要 25 秒.停止.

This is a deliberately wrong audit, it takes about 25 secs. to stop.

from subprocess import Popen, PIPE

command = "bully mon0 -e ESSID -c 8 -b aa:bb:cc:dd:ee:00 -v 2"

p = Popen(command, shell=True, stdout=PIPE, stderr=PIPE)

for line in iter(p.stdout.readline, b''):
    print "inside loop"
    print line

print "outside loop"
p.stdout.close()
p.wait()


#$ sudo python SCRIPT.py
                                ### <= 25 secs later......
# inside loop
#[!] Bully v1.0-21 - WPS vulnerability assessment utility

#inside loop
#[!] Using 'ee:cc:bb:aa:bb:ee' for the source MAC address

#inside loop
#[X] Unable to get a beacon from the AP, possible causes are

#inside loop
#[.]    an invalid --bssid or -essid was provided,

#inside loop
#[.]    the access point isn't on channel '8',

#inside loop
#[.]    you aren't close enough to the access point.

#outside loop

改用这个方法:由于输出中的大延迟和超时，我不得不修复孩子，并添加了一些技巧，所以最终代码看起来像这样

Using this method instead: Due to large delays and timeouts in the output, I had to fix the child, and added some hacks, so final code looks like this

import pexpect

child = pexpect.spawn(command)
child.maxsize = 1  #Turns off buffering
child.timeout = 50 # default is 30, insufficient for me. Crashes were due to this param.
for line in child:
    print line,

child.close()

返回相同的输出，但它实时打印行.所以......解决了谢谢@J.F.塞巴斯蒂安

Gives back the same output, but it prints lines in real time. So... SOLVED Thanks @J.F. Sebastian

推荐答案

.readlines() 读取所有行.难怪在子进程结束之前您看不到任何 输出.您可以使用 .readline() 代替在子进程刷新其标准输出缓冲区后立即逐行读取:

.readlines() reads all lines. No wonder you don't see any output until the subprocess ends. You could use .readline() instead to read line by line as soon as the subprocess flushes its stdout buffer:

from subprocess import Popen, PIPE

p = Popen("my noisy stuff", stdout=PIPE, bufsize=1)
for line in iter(p.stdout.readline, b''):
    # process line
    ..
p.stdout.close()
p.wait()

如果您已经有了 pexpect，那么您可以使用它来解决块缓冲问题:

If you are already have pexpect then you could use it to workaround the block-buffering issue:

import pexpect

child = pexpect.spawn("my noisy stuff", timeout=None)
for line in child: 
    # process line
    ..
child.close()

另见stdbuf、基于pty的解决方案，来自我在评论中链接的问题.

See also stdbuf, pty -based solutions from the question I've linked in the comments.

这篇关于解析 pexpect 输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！