如何在不使用communication()的情况下避免子进程中的死锁 [英] How to avoid the deadlock in a subprocess without using communicate()
问题描述
proc = subprocess.Popen(['start'],stdin=subprocess.PIPE,stdout=subprocess.PIPE)
proc.stdin.write('issue commands')
proc.stdin.write('issue more commands')
output = proc.stdout.read() # Deadlocked here
# Actually I have more commands to issue here
我知道communication()可以给我解决此问题的方法，但是我想稍后再发布更多命令.但是communication()会关闭子进程.
I know that communicate() can give me a solution to this problem but I wanted to issue more commands later. But communicate() kind of closes the subprocess.
在这里是否有任何已知的解决方法.我正在尝试使用python包装器与路由器进行交互.所以我有更多的命令来，这也可能会产生一些输出.在这种情况下，如何在不终止子流程的情况下进行阅读.
Is there any know work around here for this. I am trying to interact with a router using a python wrapper. So I have more commands coming up which also could produce some output. In that situation how can I read without terminating the subprocess.
推荐答案
output = proc.stdout.read() # Deadlocked here
该行会导致死锁，因为read()直到读取EOF才会返回，而EOF是在另一端关闭其stdout(例如，子进程终止)时发送的.相反，您想读取面向行的输入:
That line causes a deadlock because read() won't return until it reads EOF, which is sent when the other side closes its stdout (e.g. when the subprocess terminates). Instead, you want to read line oriented input:
output = proc.stdout.readline()
readline()将在读取换行符(或EOF)之后(即readline()读取 line 之后)返回.
readline() will return after a newline (or EOF) is read i.e. after readline() reads a line.
您的下一个僵局将由以下任一原因导致:
Your next deadlock will result from either:
-
在发送到子流程的输出中不添加
newline－当子流程尝试读取面向行的输入时，即子流程在从stdin读取时正在寻找换行符.
Not adding a
newlineto the output you send to a subprocess--when the subprocess is trying to read line oriented input, i.e. the subprocess is looking for a newline while reading from stdin.
不是flushing输出，这意味着另一端从未看到任何要读取的数据，因此另一端挂起等待数据.
Not flushing the output, which means the other side never sees any data to read, so the other side hangs waiting for data.
为了提高效率，当您写入文件(包括stdout，stdin)时，输出会被缓冲[em] ，这意味着python会欺骗您并将输出存储在以下位置中，而不是将输出实际写入文件中:列表(称为 buffer ).然后，当列表增长到一定大小时，python一次将所有输出写入文件，这比一次写一行效率更高.
For efficiency, when you write to a file (which includes stdout,stdin) output is buffered, which means instead of actually writing the output to a file, python tricks you and stores the output in a list (known as a buffer). Then when the list grows to a certain size, python writes all the output to the file at once, which is more efficient than writing a line at a time.
不容易在发送到子流程的所有输出中添加换行符就可以很容易地纠正；但是，发现需要刷新缓冲区的所有位置更加困难.这是一个示例:
Not adding a newline to all the output that you send to a subprocess can be corrected easily enough; however, discovering all the places where you need to flush buffers is more difficult. Here's an example:
prog.py:
#!/usr/bin/env python3.4 -u
import sys
print('What is your name?')
name = input()
print(name)
print("What is your number?")
number = input()
print(number)
假设您要使用另一个程序来驱动该程序?
Suppose you want to drive that program with another program?
-
使
prog.py可执行文件:$ chmod a+x prog.py
注意shebang行.
Note the shebang line.
在shebang行中请注意python解释器的-u flag，这意味着 程序的所有输出都将不被缓冲，即将其直接写入stdout- -不累积在缓冲区中.
Note the -u flag for the python interpreter in the shebang line, which means all the output for that program will be unbuffered, i.e. it will be written directly to stdout--not accumulated in a buffer.
import subprocess as sp
child = sp.Popen(
['./prog.py'],
stdin = sp.PIPE,
stdout = sp.PIPE
)
print_question = child.stdout.readline().decode('utf-8') #PIPE's send a bytes type,
#so convert to str type
name = input(print_question)
name = "{}\n".format(name)
child.stdin.write(
name.encode('utf-8') #convert to bytes type
)
child.stdin.flush()
print_name = child.stdout.readline().decode('utf-8')
print("From client: {}".format(name))
print_question = child.stdout.readline().decode('utf-8')
number = input(print_question)
number = "{}\n".format(number)
child.stdin.write(
number.encode('utf-8')
)
child.stdin.flush()
print_number = child.stdout.readline().decode('utf-8')
print("From client: {}".format(print_number))
回复评论:
您可能会遭受缓冲.大多数程序会缓冲输出以提高效率.此外，某些程序将尝试确定其stdout是否已连接至终端-如果已连接，则该程序将使用 line buffering ，这意味着从缓冲区中检索输出并将其实际写入stdout程序每次输出换行符.这样一来，使用连接的终端的人就可以与程序进行交互.
You could be suffering from buffering. Most programs buffer output for efficiency. Furthermore, some programs will try to determine if their stdout is connected to a terminal--if it is, then the program employs line buffering, which means that output is retrieved from the buffer and actually written to stdout every time the program outputs a newline. That allows a human using the connected terminal to interact with the program.
另一方面，如果程序的stdout未连接到终端，则程序将 block buffer ，这意味着仅在缓冲区具有增长到特定大小，例如4K数据.如果程序正在块缓冲中并且其输出小于4K，则实际上没有任何内容写入stdout.块缓冲允许其他计算机程序以更高的效率检索程序的输出.
On the other hand, if the program’s stdout is not connected to a terminal, the program will block buffer, which means output is retrieved from the buffer and actually written to stdout only after the buffer has grown to a specific size, say 4K of data. If the program is block buffering and it outputs less than 4K, then nothing actually gets written to stdout. Block buffering allows other computer programs to retrieve the output of the program with better efficiency.
如果路由器程序正在块缓冲中，并且其输出小于块大小，则实际上没有任何内容写入stdout，因此您的python程序没有任何内容可读取.
If the router program is block buffering, and it outputs less than the block size, then nothing actually gets written to stdout, so there is nothing for your python program to read.
您的python程序不是终端，因此路由器程序可能是块缓冲.正如JF Sebastian在评论中指出的那样，有多种方法可以诱骗路由器程序以使您的python程序是终端机，这将导致路由器程序 line buffer ，因此您将能够用readline()从其标准输出读取.查看pexpect.
Your python program is not a terminal, so the router program may be block buffering. As J.F. Sebastian pointed out in the comments, there are ways to trick the router program into thinking your python program is a terminal, which will cause the router program to line buffer, and therefore you will be able to read from its stdout with readline(). Check out pexpect.
这篇关于如何在不使用communication()的情况下避免子进程中的死锁的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
