Python子进程Popen.communicate()是否等效于Popen.stdout.read()? [英] Python subprocess Popen.communicate() equivalent to Popen.stdout.read()?

问题描述

非常具体的问题(希望如此):以下三个代码之间有什么区别?

Very specific question (I hope): What are the differences between the following three codes?

(我希望只有第一个不会等待子进程完成，而第二个和第三个会完成.但是我需要确保这是 only 的区别...)

(I expect it to be only that the first does not wait for the child process to be finished, while the second and third ones do. But I need to be sure this is the only difference...)

我也欢迎其他评论/建议(尽管我已经很清楚shell=True的危险和跨平台限制)

I also welcome other remarks/suggestions (though I'm already well aware of the shell=True dangers and cross-platform limitations)

请注意，我已经阅读了 Python子进程交互，为什么我的流程可以与Popen.communicate一起使用，而不能与Popen.stdout.read()一起使用?，并且我以后不想/不需要与程序进行交互.

Note that I already read Python subprocess interaction, why does my process work with Popen.communicate, but not Popen.stdout.read()? and that I do not want/need to interact with the program after.

还要注意，我已经阅读了 Python Popen.communicate()内存的替代方法限制?但是我没有真正理解它...

Also note that I already read Alternatives to Python Popen.communicate() memory limitations? but that I didn't really get it...

最后，请注意，我知道在某个地方，当使用一种方法将一个缓冲区填充一个输出时，会有死锁的风险，但是我在互联网上寻找清晰的解释时迷路了...

Finally, note that I am aware that somewhere there is a risk of deadlock when one buffer is filled with one output using one method, but I got lost while looking for clear explanations on the Internet...

第一个代码:

from subprocess import Popen, PIPE

def exe_f(command='ls -l', shell=True):
    """Function to execute a command and return stuff"""

    process = Popen(command, shell=shell, stdout=PIPE, stderr=PIPE)

    stdout = process.stdout.read()
    stderr = process.stderr.read()

    return process, stderr, stdout

第二个代码:

from subprocess import Popen, PIPE
from subprocess import communicate

def exe_f(command='ls -l', shell=True):
    """Function to execute a command and return stuff"""

    process = Popen(command, shell=shell, stdout=PIPE, stderr=PIPE)

    (stdout, stderr) = process.communicate()

    return process, stderr, stdout

第三个代码:

from subprocess import Popen, PIPE
from subprocess import wait

def exe_f(command='ls -l', shell=True):
    """Function to execute a command and return stuff"""

    process = Popen(command, shell=shell, stdout=PIPE, stderr=PIPE)

    code   = process.wait()
    stdout = process.stdout.read()
    stderr = process.stderr.read()

    return process, stderr, stdout

谢谢.

推荐答案

如果您查看subprocess.communicate()的源代码，它将显示一个完美的区别示例:

If you look at the source for subprocess.communicate(), it shows a perfect example of the difference:

def communicate(self, input=None):
    ...
    # Optimization: If we are only using one pipe, or no pipe at
    # all, using select() or threads is unnecessary.
    if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
        stdout = None
        stderr = None
        if self.stdin:
            if input:
                self.stdin.write(input)
            self.stdin.close()
        elif self.stdout:
            stdout = self.stdout.read()
            self.stdout.close()
        elif self.stderr:
            stderr = self.stderr.read()
            self.stderr.close()
        self.wait()
        return (stdout, stderr)

    return self._communicate(input)

您可以看到communicate确实利用了对stdout和stderr的读取调用，并且还调用了wait().这只是操作顺序的问题.在您的情况下，因为您同时将stdout和stderr都使用PIPE，它将进入_communicate():

You can see that communicate does make use of the read calls to stdout and stderr, and also calls wait(). It is just a matter of order of operations. In your case because you are using PIPE for both stdout and stderr, it goes into _communicate():

def _communicate(self, input):
    stdout = None # Return
    stderr = None # Return

    if self.stdout:
        stdout = []
        stdout_thread = threading.Thread(target=self._readerthread,
                                         args=(self.stdout, stdout))
        stdout_thread.setDaemon(True)
        stdout_thread.start()
    if self.stderr:
        stderr = []
        stderr_thread = threading.Thread(target=self._readerthread,
                                         args=(self.stderr, stderr))
        stderr_thread.setDaemon(True)
        stderr_thread.start()

    if self.stdin:
        if input is not None:
            self.stdin.write(input)
        self.stdin.close()

    if self.stdout:
        stdout_thread.join()
    if self.stderr:
        stderr_thread.join()

    # All data exchanged.  Translate lists into strings.
    if stdout is not None:
        stdout = stdout[0]
    if stderr is not None:
        stderr = stderr[0]

    # Translate newlines, if requested.  We cannot let the file
    # object do the translation: It is based on stdio, which is
    # impossible to combine with select (unless forcing no
    # buffering).
    if self.universal_newlines and hasattr(file, 'newlines'):
        if stdout:
            stdout = self._translate_newlines(stdout)
        if stderr:
            stderr = self._translate_newlines(stderr)

    self.wait()
    return (stdout, stderr)

这使用线程一次读取多个流.然后它在最后调用wait().

This uses threads to read from multiple streams at once. Then it calls wait() at the end.

所以总结一下:

1. 此示例一次读取一个流，而无需等待它完成该过程.
2. 此示例通过内部线程同时从两个流中读取，并等待它完成该过程.
3. 此示例等待过程完成，然后一次读取一个流.正如您提到的，如果流中写入的内容过多，则有可能导致死锁.

此外，在第二个和第三个示例中，您不需要这两个import语句:

Also, you don't need these two import statements in your 2nd and 3rd examples:

from subprocess import communicate
from subprocess import wait

它们都是Popen对象的方法.

这篇关于Python子进程Popen.communicate()是否等效于Popen.stdout.read()?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！