Python 子进程 readlines() 挂起 [英] Python subprocess readlines() hangs

问题描述

我尝试完成的任务是流式传输 ruby​​ 文件并打印输出.(注意:我不想一次打印出所有内容)

The task I try to accomplish is to stream a ruby file and print out the output. (NOTE: I don't want to print out everything at once)

ma​​in.py

from subprocess import Popen, PIPE, STDOUT

import pty
import os

file_path = '/Users/luciano/Desktop/ruby_sleep.rb'

command = ' '.join(["ruby", file_path])

master, slave = pty.openpty()
proc = Popen(command, bufsize=0, shell=True, stdout=slave, stderr=slave, close_fds=True)     
stdout = os.fdopen(master, 'r', 0)

while proc.poll() is None:
    data = stdout.readline()
    if data != "":
        print(data)
    else:
        break

print("This is never reached!")

ruby_sleep.rb

puts "hello"

sleep 2

puts "goodbye!"

问题

流式传输文件工作正常.hello/goodbye 输出以 2 秒的延迟打印.就像脚本应该工作一样.问题是 readline() 最后挂起并且永不退出.我永远不会到达最后一张照片.

Streaming the file works fine. The hello/goodbye output is printed with the 2 seconds delay. Exactly as the script should work. The problem is that readline() hangs in the end and never quits. I never reach the last print.

我知道这里有很多这样的问题 stackoverflow 但没有一个问题让我解决了这个问题.我不太了解整个子流程，所以请给我一个更实际/具体的答案.

I know there is a lot of questions like this here a stackoverflow but non of them made me solve the problem. I'm not that into the whole subprocess thing so please give me a more hands-on/concrete answer.

问候

编辑

修复意外代码.(与实际错误无关)

Fix unintended code. (nothing to do with the actual error)

推荐答案

由于 问:为什么不直接使用管道 (popen())?(到目前为止，所有其他答案都忽略了您的注意:我不"不想一次打印出所有内容").

I assume you use pty due to reasons outlined in Q: Why not just use a pipe (popen())? (all other answers so far ignore your "NOTE: I don't want to print out everything at once").

pty 仅适用于 Linux 如文档中所述:

pty is Linux only as said in the docs:

因为伪终端处理高度依赖平台，所以有是仅适用于 Linux 的代码.(Linux 代码应该可以工作在其他平台上，但尚未经过测试.)

Because pseudo-terminal handling is highly platform dependent, there is code to do it only for Linux. (The Linux code is supposed to work on other platforms, but hasn’t been tested yet.)

尚不清楚它在其他操作系统上的运行情况.

It is unclear how well it works on other OSes.

你可以试试pexpect:

import sys
import pexpect

pexpect.run("ruby ruby_sleep.rb", logfile=sys.stdout)

或 stdbuf 在非交互模式下启用行缓冲:

Or stdbuf to enable line-buffering in non-interactive mode:

from subprocess import Popen, PIPE, STDOUT

proc = Popen(['stdbuf', '-oL', 'ruby', 'ruby_sleep.rb'],
             bufsize=1, stdout=PIPE, stderr=STDOUT, close_fds=True)
for line in iter(proc.stdout.readline, b''):
    print line,
proc.stdout.close()
proc.wait()

或者根据 @Antti Haapala 的回答使用 stdlib 中的 pty :

Or using pty from stdlib based on @Antti Haapala's answer:

#!/usr/bin/env python
import errno
import os
import pty
from subprocess import Popen, STDOUT

master_fd, slave_fd = pty.openpty()  # provide tty to enable
                                     # line-buffering on ruby's side
proc = Popen(['ruby', 'ruby_sleep.rb'],
             stdin=slave_fd, stdout=slave_fd, stderr=STDOUT, close_fds=True)
os.close(slave_fd)
try:
    while 1:
        try:
            data = os.read(master_fd, 512)
        except OSError as e:
            if e.errno != errno.EIO:
                raise
            break # EIO means EOF on some systems
        else:
            if not data: # EOF
                break
            print('got ' + repr(data))
finally:
    os.close(master_fd)
    if proc.poll() is None:
        proc.kill()
    proc.wait()
print("This is reached!")

所有三个代码示例都立即打印hello"(一旦看到第一个 EOL).

All three code examples print 'hello' immediately (as soon as the first EOL is seen).

^{将旧的更复杂的代码示例留在这里，因为它可能会在 SO 的其他帖子中被引用和讨论}

或者使用基于 @Antti Haapala 的回答的pty:

import os
import pty
import select
from subprocess import Popen, STDOUT

master_fd, slave_fd = pty.openpty()  # provide tty to enable
                                     # line-buffering on ruby's side
proc = Popen(['ruby', 'ruby_sleep.rb'],
             stdout=slave_fd, stderr=STDOUT, close_fds=True)
timeout = .04 # seconds
while 1:
    ready, _, _ = select.select([master_fd], [], [], timeout)
    if ready:
        data = os.read(master_fd, 512)
        if not data:
            break
        print("got " + repr(data))
    elif proc.poll() is not None: # select timeout
        assert not select.select([master_fd], [], [], 0)[0] # detect race condition
        break # proc exited
os.close(slave_fd) # can't do it sooner: it leads to errno.EIO error
os.close(master_fd)
proc.wait()

print("This is reached!")

这篇关于Python 子进程 readlines() 挂起的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！