定时一个外部程序，其输出正在由Python处理 [英] Time an external program whose output is being processed by Python

问题描述

我想测量一个外部程序的执行时间，该程序的输出被我的Python脚本使用.

I want to measure the time of execution of an external program whose output is used by my Python script.

调用extprogram生成输出的程序，此刻我执行类似的操作:

Calling extprogram the program that produced the output, at the moment I do something like:

import time
import subprocess

def process_output(line):
   ...
   ...
   return processed_data

all_processed_data = []

ts = time.time()
p = subprocess.Popen("extprogram", stdout=subprocess.PIPE)

for line in p.stdout:
    all_processed_data.append(process_output(line))
te = time.time()
elapsed_time = te - ts

这不符合预期，因为我要测量的是extprogram的执行时间加上处理其输出所需的时间.

This doesn't work as intended because what I am measuring is the time of execution of extprogram plus the time required to process its output.

extprogram产生大量数据，因此我想像现在一样使用循环在"Python"程序中流式传输"其输出. 当extprogram终止而不是等待所有输出被处理时，如何评估te?

extprogram produces a large amount of data, therefore I would like to "stream" its output in my Python program using a cycle as I am doing now. How can I evaluate te when extprogram terminates rather than waiting for all the output to be processed?

推荐答案

以下内容仍使用挂钟"时间，但可以替代使用主机系统时间命令.执行和计时分为单独的线程，可以在执行任何处理之前停止计时器.

The following still uses 'wall clock' time but may be an alternative to the use of host system time commands. The execution and the timing are split into separate threads and the timer can be stopped before any processing is carried out.

from multiprocessing import Event
import threading
import time
import subprocess

def timing(event):
    print "timer starts"
    ts = time.time()
    event.wait()
    te = time.time()
    elapsed_time = te - ts
    print "Elapsed Time " + str(elapsed_time)

def execution(event): 
    for i in range(0,1000):
        p = subprocess.Popen("ls", stdout=subprocess.PIPE)
    event.set()

if __name__ == '__main__':  
    event = Event()
    e = threading.Thread(target=execution, args=(event,))
    t = threading.Thread(target=timing, args=(event,))
    t.start()  
    e.start() 
    while not event.is_set():
        print "running..."
        time.sleep(1)

这给了我以下输出:

timer starts
running...
running...
Elapsed Time 1.66236400604

或者您可以从输出处理中分离接收"extprogram"的输出.

Or you could split receiving the output of 'extprogram' from the processing of the output.

例如:

ts = time.time()
p = subprocess.Popen("extprogram", stdout=subprocess.PIPE)

for line in p.stdout:
    tempdata.append(line)

te = time.time()
elapsed_time = te - ts

for line in tempdata:
    all_processed_data.append(process_output(line))

这篇关于定时一个外部程序，其输出正在由Python处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！