当使用“时间"时访问标准输出.在python子过程中 [英] Accessing stdout when using "time" in python subproces
问题描述
我一直在使用time命令在我的shell中进行一些手动基准测试.我想通过编写一个Python脚本来扩展基准测试，该脚本既可以自动执行测试，又可以让我访问时间数据，以便可以选择的格式(可能是csv)记录下来.我看到有一个timeit模块，但这似乎更多是用于对python代码进行基准测试，在这里我要进行基准测试的是在命令行中运行的程序.
I have been doing some manual benchmark tests in my shell using the time command. I would like to scale my benchmarks by writing a python script that both automates the tests and affords me access to the time data so that I can record it in the format of my choosing (likely a csv). I see there is the timeit module, but that seems like it is more for benchmarking python code, where what I am trying to benchmark here are programs run in the command line.
这是我一直在手动执行的操作:
This is what I have been doing manually:
time program -aflag -anotherflag
我最初在脚本中实现此目标的尝试如下:
My initial attempt to implement this in a script looks like:
cmnd = ['time', 'program', 'aflag', 'anotherflag']
p = subprocess.Popen(cmnd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate
print out
print err
我可以很好地访问time的输出–它已传递到stderr，但是我没有在stdout中获得program的输出.如果我从cmnd中删除time并将shell=False更改为True，则可以在stdout中获得程序的输出-但是显然不是time的输出，这就是重点.
I can access the output of time just fine – this is delivered to stderr, however I am not getting program's output as expected in stdout. If I remove time from cmnd and change shell=False to True, I then get the program's output in stdout – however obviously not time's output, which is the whole point.
cmnd = ['program', 'aflag', 'anotherflag']
p = subprocess.Popen(cmnd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate
print out
print err
如果使用shell=True将time添加回cmnd，我会得到时间输出，但是program实际上并没有运行.
If I add time back to cmnd with shell=True, I get time's output but program doesn't actually run.
我怎么都可以工作?
推荐答案
Instead of trying to get this to work, why not use the functionality built into Python in the resource module?
import resource
import subprocess
cmd = ['program', 'aflag', 'anotherflag']
p = subprocess.Popen(cmd, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
usage = resource.getrusage(resource.RUSAGE_CHILDREN)
print out
print err
print usage.ru_utime, usage.ru_stime, usage.ru_utime+usage.ru_stime
如果您需要区分同时运行的不同子进程，那么getrusage显然还不够.在这种情况下，您需要使用 wait4 或类似的方法获取每个进程的资源使用情况.这使Popen的使用更加复杂.在这种情况下，您可能想要做的是子类或派发subprocess代码(但请确保使用 subprocess32 反向移植，如果您使用的是3.1或更早版本，则可以避免communicate中的错误-以便该类实际上具有您要挂钩的方法…)并更改
If you need to distinguish different child processes running simultaneously, getrusage isn't obviously not sufficient. In that case, you need to use wait4 or similar to get per-process resource usage. This makes your use of Popen more complicated. What you'd probably want to do for this case is subclass or fork the subprocess code (but make sure to use subprocess32 backport if you're on 3.1 or earlier to avoid the bugs in communicate—and so that the class actually has the method you want to hook…) and change the _try_wait method to use wait4 instead of waitpid and stash the extra results in, e.g., self.rusage so you can access it later.
我认为类似的方法会起作用:
I think something like this would work:
import subprocess32
class Popen(subprocess32.Popen):
def _try_wait(self, wait_flags):
"""All callers to this function MUST hold self._waitpid_lock."""
try:
(pid, sts, rusage) = _eintr_retry_call(os.wait4, self.pid, wait_flags)
if pid == self.pid:
self.rusage = rusage
except OSError as e:
if e.errno != errno.ECHILD:
raise
pid = self.pid
sts = 0
return (pid, sts)
cmd = ['program', 'aflag', 'anotherflag']
p = Popen(cmd, shell=False, stdout=subprocess32.PIPE, stderr=subprocess32.PIPE)
out, err = p.communicate()
print out
print err
try:
usage = p.rusage
except AttributeError:
print 'Child died before we could wait on it, no way to get rusage'
else:
print usage.ru_utime, usage.ru_stime, usage.ru_utime+usage.ru_stime
这篇关于当使用“时间"时访问标准输出.在python子过程中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
