带有/usr/bin/time 的 Python 子进程:如何捕获计时信息，但忽略所有其他输出? [英] Python subprocess with /usr/bin/time: How can I capture timing information, but ignore all other output?

问题描述

我正在尝试测量通过子进程调用的可执行程序的执行时间(以秒为单位).我不希望发出可执行文件(stderr 或 stdout)的输出.

I am trying to measure the execution time in seconds of an executable program invoked via subprocess. I do not want the output of the executable (either stderr or stdout) to be emitted.

尝试过timeit和resource库，都没有准确的捕捉到进程的时间，貌似只能捕捉到Python工作线程中的时间.

I have tried the timeit and resource libraries, but neither accurately captures the time of the process, seemingly it only captures the timing in the Python worker thread.

下面的这种尝试将丢失 stderr 重定向的时间信息 b/c.但是，没有 stderr 重定向，将发出命令 'f_cmd' stderr 输出.

This attempt below will lose the timing info b/c of the stderr redirect. However, w/o the stderr redirect, the command 'f_cmd' stderr output will be emitted.

def doWithTiming(f_cmd):
    DEVNULL = open(os.devnull, 'w')
    return subprocess.check_output([ "/usr/bin/time", "--format=%e seconds"] + f_cmd.split(), stderr=DEVNULL)

如何忽略 f_cmd 的所有输出但保留/usr/bin/time 的输出?

How do I ignore all output of f_cmd but retain the output of /usr/bin/time?

推荐答案

%e /usr/bin/time 格式为:

进程使用的实际(挂钟)时间，以秒为单位.

Elapsed real (wall clock) time used by the process, in seconds.

使用抑制的 stdout/stderr 运行子进程并获取经过的时间:

To run a subprocess with suppressed stdout/stderr and get the elapsed time:

#!/usr/bin/env python
import os
import time
from subprocess import check_call, STDOUT

DEVNULL = open(os.devnull, 'wb', 0)

start = time.time()
check_call(['sleep', '1'], stdout=DEVNULL, stderr=STDOUT)
print("{:.3f} seconds".format(time.time() - start))

<小时>

timeit.default_timer 在 Python 2 上的 POSIX 上是 time.time 因此你应该有一个有效的时间，除非你对 timeit 的使用是不正确.

---

timeit.default_timer is time.time on POSIX on Python 2 therefore you should have got a valid time unless your usage of timeit is incorrect.

resource 模块返回的信息不包括真实"时间，但您可以使用它来获取用户"和系统"时间，即，进程在用户模式下花费的 CPU 秒总数." 和进程在内核模式下花费的 CPU 秒总数." 相应地:>

The information returned by resource module does not include the "real" time, but you could use it to get "user" and "sys" times i.e., "Total number of CPU-seconds that the process spent in user mode." and "Total number of CPU-seconds that the process spent in kernel mode." correspondingly:

#!/usr/bin/env python
import os
import time
from subprocess import Popen, STDOUT

DEVNULL = open(os.devnull, 'wb', 0)

start = time.time()
p = Popen(['sleep', '1'], stdout=DEVNULL, stderr=STDOUT)
ru = os.wait4(p.pid, 0)[2]
elapsed = time.time() - start
print(" {:.3f}real {:.3f}user {:.3f}system".format(
       elapsed, ru.ru_utime, ru.ru_stime))

<小时>

您可以使用 psutil.Popen 启动子进程，并在子进程运行时获取附加信息(cpu、内存、网络连接、线程、fds、子进程、等)以便携的方式.

---

You could start a subprocess using psutil.Popen and get while the child process is running additional info (cpu, memory, network connections, threads, fds, children, etc) in a portable way.

另请参阅如何在 Python 中使用 psutil 获取程序的最大内存使用量.

为了测试(以确保基于 time.time() 的解决方案产生相同的结果)，您可以捕获 /usr/bin/time 输出:

For testing (to make sure that time.time()-based solution produces the same results), you could capture /usr/bin/time output:

#!/usr/bin/env python
import os
from collections import deque
from subprocess import Popen, PIPE

DEVNULL = open(os.devnull, 'wb', 0)

time_lines_count = 1 # how many lines /usr/bin/time produces
p = Popen(['/usr/bin/time', '--format=%e seconds'] + 
          ['sleep', '1'], stdout=DEVNULL, stderr=PIPE)
with p.stderr:
    q = deque(iter(p.stderr.readline, b''), maxlen=time_lines_count)
rc = p.wait()
print(b''.join(q).decode().strip())

或者使用带有命名管道的 -o 选项:

Or using -o option with a named pipe:

#!/usr/bin/env python
import os
from contextlib import contextmanager
from shutil     import rmtree
from subprocess import Popen, STDOUT
from tempfile   import mkdtemp

DEVNULL = open(os.devnull, 'wb', 0)

@contextmanager
def named_pipe():
    dirname = mkdtemp()
    try:
        path = os.path.join(dirname, 'named_pipe')
        os.mkfifo(path)
        yield path
    finally:
        rmtree(dirname)

with named_pipe() as path:
    p = Popen(['/usr/bin/time', '--format=%e seconds', '-o', path] + 
              ['sleep', '1'], stdout=DEVNULL, stderr=STDOUT)
    with open(path) as file:
        time_output = file.read().strip()
    rc = p.wait()
print(time_output)

这篇关于带有/usr/bin/time 的 Python 子进程:如何捕获计时信息，但忽略所有其他输出?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！