Python子进程模块比命令慢得多（不推荐） [英] Python subprocess module much slower than commands (deprecated)

问题描述

所以我写了一个脚本，在命令行中使用nc访问一系列服务器，最初我使用的是Python的命令模块并调用commands.getoutput（），脚本在大约45秒内运行。由于命令已弃用，我想将所有内容更改为使用subprocess模块​​，但现在脚本需要2m45s才能运行。

我以前有什么：

  output = commands.getoutput（echo get file.ext | nc -w 1 server.com port_num）
  

b $ b

现在我有

  p = Popen（'echo get file.ext | nc -w 1 server。 com port_num'，shell = True，stdout = PIPE）
 output = p.communicate（）[0] 
  
 
 
 
先感谢您的帮助！
解决方案
我会期望 subprocess 要慢于命令。没有意义表明这只是 原因，您的脚本运行缓慢，你应该看看 命令 源代码。有少于100行，大多数工作委托给来自 os 的函数，其中许多直接从c posix库（至少在posix系统中）。请注意，命令是仅限unix的，因此它不必做任何额外的工作，以确保跨平台兼容性。 
 
 
 
现在，请查看  subprocess  。有超过1500行，所有纯Python，做各种检查，以确保一致的跨平台行为。基于这个，我会期望 subprocess 比命令运行得慢。 
 
 
 
我计时了两个模块，在一些很基本的东西， subprocess 几乎是 commands 。 
 >>> ％timeit commands.getoutput（'echofoo| cat'）
 100循环，最好的3：每个循环3.02 ms 
>>> ％timeit subprocess.check_output（'echofoo| cat'，shell = True）
 100循环，最好是3：每循环5.76 ms 
  
  Swiss 提供了一些有助于改善您的脚本性能的改进。但即使在应用它们后，请注意 subprocess 仍然较慢。 
 >>> ％timeit commands.getoutput（'echofoo| cat'）
 100循环，最好的3：每个循环2.97 ms 
>>> ％timeit Popen（'cat'，stdin = PIPE，stdout = PIPE）.communicate（'foo'）[0] 
 100循环，最好的3：每循环4.15 ms 
  
假设你连续多次执行上面的命令，这将累加起来，并至少解决一些性能差异。 
 
 
 
无论如何，我解释你的问题是关于 subprocess 和命令，而不是关于如何加快你的脚本。对于后一个问题，瑞士的答案更好。
 
So I wrote a script that accesses a bunch of servers using nc on the command line, and originally I was using Python's commands module and calling commands.getoutput() and the script ran in about 45 seconds. Since commands is deprecated, I want to change everything over to using the subprocess module, but now the script takes 2m45s to run. Anyone have an idea of why this would be?

What I had before:
output = commands.getoutput("echo get file.ext | nc -w 1 server.com port_num")
now I have 
p = Popen('echo get file.ext | nc -w 1 server.com port_num', shell=True, stdout=PIPE)
output = p.communicate()[0]
Thanks in advance for the help!
解决方案
I would expect subprocess to be slower than command. Without meaning to suggest that this is the only reason your script is running slowly, you should take a look at the commands source code. There are fewer than 100 lines, and most of the work is delegated to functions from os, many of which are taken straight from c posix libraries (at least in posix systems). Note that commands is unix-only, so it doesn't have to do any extra work to ensure cross-platform compatibility. 

Now take a look at subprocess. There are more than 1500 lines, all pure Python, doing all sorts of checks to ensure consistent cross-platform behavior. Based on this, I would expect subprocess to run slower than commands. 

I timed the two modules, and on something quite basic, subprocess was almost twice as slow as commands. 
>>> %timeit commands.getoutput('echo "foo" | cat')
100 loops, best of 3: 3.02 ms per loop
>>> %timeit subprocess.check_output('echo "foo" | cat', shell=True)
100 loops, best of 3: 5.76 ms per loop
Swiss suggests some good improvements that will help your script's performance. But even after applying them, note that subprocess is still slower. 
>>> %timeit commands.getoutput('echo "foo" | cat')
100 loops, best of 3: 2.97 ms per loop
>>> %timeit Popen('cat', stdin=PIPE, stdout=PIPE).communicate('foo')[0]
100 loops, best of 3: 4.15 ms per loop
Assuming you are performing the above command many times in a row, this will add up, and account for at least some of the performance difference. 

In any case, I am interpreting your question as being about the relative performance of subprocess and command, rather than being about how to speed up your script. For the latter question, Swiss's answer is better.

                        
这篇关于Python子进程模块比命令慢得多（不推荐）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！