管道传输时在python的子流程模块中使用stdout.close() [英] Usage of stdout.close() in python's subprocess module when piping

问题描述

在python子进程模块中，如果我们想运行shell命令

In the python subprocess module, if we wanted to run the shell command

foo | grep bar

在python中，我们可能会使用

from within python, we might use

p1 = Popen(["foo"], stdout = PIPE)
p2 = Popen(["grep", "bar"], stdin = p1.stdout, stdout = PIPE)
p1.stdout.close()
output = p2.communicate()[0]

我对p1.stdout.close()行感到困惑.如果您能原谅我，那么我将跟踪我认为该程序的工作方式，并且该错误有望揭露.

I'm confused about the line p1.stdout.close(). If you'll forgive me, I'll trace through how I think the program works, and the error will hopefully reveal itself.

在我看来，当python颁布output = p2.communicate()[0]行时，python尝试调用p2，它识别出它需要从p1输出.因此，它调用p1，它执行foo并将输出扔到堆栈上，以便p2可以完成执行.然后p2完成.

It seems to me that when the line output = p2.communicate()[0] is enacted by python, python tries to call p2, it recognizes that it needs output from p1. So it calls p1, which executes foo and throws the output on the stack so that p2 can finish executing. And then p2 finishes.

但是p1.stdout.close()实际上没有出现在此跟踪中的任何地方.那么实际发生了什么?在我看来，行的排序可能也很重要，因此以下操作将无效:

But nowhere in this trace does p1.stdout.close() actually happen. So what is actually happening? It seems to me that this ordering of lines might matter too, so that the following wouldn't work:

p1 = Popen(["foo"], stdout = PIPE)
p1.stdout.close()
p2 = Popen(["grep", "bar"], stdin = p1.stdout, stdout = PIPE)
output = p2.communicate()[0]

这就是我的理解状态.

推荐答案

p1.stdout.close()对于foo检测管道何时断开(例如，p2过早退出)是必需的.

p1.stdout.close() is necessary for foo to detect when the pipe is broken e.g., when p2 exits prematurely.

如果没有p1.stdout.close()，则p1.stdout在父进程中保持打开状态，即使p2退出； p1不会知道没有人读取p1.stdout，即，p1将继续写入p1.stdout，直到相应的OS管道缓冲区已满，然后永远阻塞为止.

If there is no p1.stdout.close() then p1.stdout remains open in the parent process and even if p2 exits; p1 won't know that nobody reads p1.stdout i.e., p1 will continue to write to p1.stdout until the corresponding OS pipe buffer is full and then it just blocks forever.

要模拟没有外壳的foo | grep bar shell命令:

To emulate foo | grep bar shell command without a shell:

#!/usr/bin/env python3
from subprocess import Popen, PIPE

with Popen(['grep', 'bar'], stdin=PIPE) as grep, \
     Popen(['foo'], stdout=grep.stdin):
    grep.communicate()

请参见如何使用subprocess.Popen通过管道连接多个进程?

这篇关于管道传输时在python的子流程模块中使用stdout.close()的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！