自定义 Popen.communicate 方法给出错误的输出 [英] Custom Popen.communicate method gives wrong output

问题描述

让我们从考虑这段代码开始:

Let's start by considering this code:

proc_stdin.py

import sys

if __name__ == '__main__':
    for i, line in enumerate(sys.stdin):
        sys.stdout.write(line)

test.py

import subprocess


def run_bad(target, input=None):
    proc = subprocess.Popen(
        target,
        universal_newlines=True,
        shell=True,
        stderr=subprocess.STDOUT,
        stdin=subprocess.PIPE if input else subprocess.DEVNULL,
        stdout=subprocess.PIPE,
    )

    if input:
        proc.stdin.write(input)
        proc.stdin.flush()
        proc.stdin.close()

    lines = []
    for line in iter(proc.stdout.readline, ""):
        line = line.rstrip("\n")
        lines.append(line)
    proc.stdout.close()

    ret_code = proc.wait()
    return "\n".join(lines)


def run_good(target, input):
    return subprocess.Popen(
        target,
        universal_newlines=True,
        shell=True,
        stderr=subprocess.STDOUT,
        stdin=subprocess.PIPE,
        stdout=subprocess.PIPE,
    ).communicate(input=input)[0]


if __name__ == '__main__':
    lst = [
        "",
        "token1",
        "token1\n",
        "token1\r\n",
        "token1\n\n",
        "token1\r\n\ntoken2",
        "token1 token2",
        "token1\ntoken2",
        "token1\r\ntoken2",
        "token1\n\ntoken2",
        "token1\r\n\ntoken2",
        "token1 \ntoken2\ntoken2\n"
    ]
    cmd = "python proc_stdin.py"

    for inp in lst:
        a, b = run_bad(cmd, inp), run_good(cmd, inp)
        if a != b:
            print("Error: {} vs {}".format(repr(a), repr(b)))
        else:
            print("ok: {}".format(repr(a)))

输出:

ok: ''
ok: 'token1'
Error: 'token1' vs 'token1\n'
Error: 'token1\n' vs 'token1\n\n'
Error: 'token1\n' vs 'token1\n\n'
ok: 'token1\n\n\ntoken2'
ok: 'token1 token2'
ok: 'token1\ntoken2'
ok: 'token1\n\ntoken2'
ok: 'token1\n\ntoken2'
ok: 'token1\n\n\ntoken2'
Error: 'token1 \ntoken2\ntoken2' vs 'token1 \ntoken2\ntoken2\n'

我的问题是，为什么 run_bad 和run_good 不是在所有情况下都相等?您将如何更改 run_bad 函数以使输出等于 run_good?

My question is, why is the output of both run_bad & run_good not equal in all cases? How would you change the run_bad function so the output becomes equal than run_good?

您可能还想知道，为什么不直接使用 Popen.communicate 来处理这种特殊情况或来自 subprocess 模块的其他帮助程序?好吧，在现实世界的情况下，我正在为 SublimeText3 创建一个插件，这迫使我坚持使用 python3.3(不能使用许多现代子进程的好东西)而且我想在阅读行时注入一些回调来自标准输出，这是我无法通过使用 Popen.communicate 方法(据我所知)来做的事情.

You also may wonder, why are you not using directly Popen.communicate for this particular case or other helpers from subprocess module? Well, in the real world case I'm creating a plugin for SublimeText3 which is forcing me to stick to python3.3 (can't use many of the modern subprocess goodies) plus I'd like to inject some callbacks while reading the lines from stdout and that's something I can't do by using the Popen.communicate method (as far as I know).

提前致谢.

推荐答案

如果您从每一行中去除换行符，然后将它们添加回行之间，最后一个换行符(如果有)会发生什么?(在最后的换行符之后没有最后的空行，因为您的 iter 丢弃了它.)这就是 Python 的 readline(或行迭代)函数 includes 换行符:它们是准确表示文件结尾所必需的.

If you strip newlines from every line and then add them back between the lines, what happens to the last newline (if any)? (There’s no final, empty line after a final newline because your iter discards it.) This is why Python’s readline (or line iteration) function includes the newlines: they’re necessary to represent the end of the file accurately.

这篇关于自定义 Popen.communicate 方法给出错误的输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！