Python 子进程模块比命令慢得多(已弃用) [英] Python subprocess module much slower than commands (deprecated)

问题描述

所以我写了一个脚本，在命令行上使用 nc 访问一堆服务器，最初我使用 Python 的命令模块并调用 commands.getoutput() 并且脚本运行了大约 45 秒.由于不推荐使用命令，因此我想将所有内容更改为使用 subprocess 模块，但现在脚本需要 2m45s 才能运行.有人知道为什么会这样吗?

So I wrote a script that accesses a bunch of servers using nc on the command line, and originally I was using Python's commands module and calling commands.getoutput() and the script ran in about 45 seconds. Since commands is deprecated, I want to change everything over to using the subprocess module, but now the script takes 2m45s to run. Anyone have an idea of why this would be?

我之前的:

output = commands.getoutput("echo get file.ext | nc -w 1 server.com port_num")

现在我有

p = Popen('echo get file.ext | nc -w 1 server.com port_num', shell=True, stdout=PIPE)
output = p.communicate()[0]

提前感谢您的帮助！

推荐答案

我希望 subprocess 比 command 慢.无意暗示这是您的脚本运行缓慢的唯一原因，您应该查看 commands 源代码.不到 100 行，大部分工作都委托给来自 os 的函数，其中许多直接取自 c posix 库(至少在 posix 系统中).请注意，commands 仅适用于 unix，因此无需进行任何额外工作即可确保跨平台兼容性.

I would expect subprocess to be slower than command. Without meaning to suggest that this is the only reason your script is running slowly, you should take a look at the commands source code. There are fewer than 100 lines, and most of the work is delegated to functions from os, many of which are taken straight from c posix libraries (at least in posix systems). Note that commands is unix-only, so it doesn't have to do any extra work to ensure cross-platform compatibility.

现在看看 subprocess.有 1500 多行，全部是纯 Python，做各种检查以确保一致的跨平台行为.基于此，我希望 subprocess 运行速度比 commands 慢.

Now take a look at subprocess. There are more than 1500 lines, all pure Python, doing all sorts of checks to ensure consistent cross-platform behavior. Based on this, I would expect subprocess to run slower than commands.

我对这两个模块进行了计时，在非常基本的情况下，subprocess 的速度几乎是 commands 的两倍.

I timed the two modules, and on something quite basic, subprocess was almost twice as slow as commands.

>>> %timeit commands.getoutput('echo "foo" | cat')
100 loops, best of 3: 3.02 ms per loop
>>> %timeit subprocess.check_output('echo "foo" | cat', shell=True)
100 loops, best of 3: 5.76 ms per loop

Swiss 提出了一些有助于提高脚本性能的改进建议.但即使在应用它们之后，请注意 subprocess 仍然 较慢.

Swiss suggests some good improvements that will help your script's performance. But even after applying them, note that subprocess is still slower.

>>> %timeit commands.getoutput('echo "foo" | cat')
100 loops, best of 3: 2.97 ms per loop
>>> %timeit Popen('cat', stdin=PIPE, stdout=PIPE).communicate('foo')[0]
100 loops, best of 3: 4.15 ms per loop

假设您连续多次执行上述命令，这将加起来，并至少解释部分性能差异.

Assuming you are performing the above command many times in a row, this will add up, and account for at least some of the performance difference.

无论如何，我将您的问题解释为关于 subprocess 和 command 的相对性能，而不是关于如何加快脚本的速度.对于后一个问题，Swiss 的答案更好.

In any case, I am interpreting your question as being about the relative performance of subprocess and command, rather than being about how to speed up your script. For the latter question, Swiss's answer is better.

这篇关于Python 子进程模块比命令慢得多(已弃用)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！