为什么在写入python子进程stdin管道时丢失数据? [英] Why is data missing when I write to a python subprocess stdin pipe?

问题描述

我的python代码如下:

My python code looks like this:

def test():
    pipe = sp.Popen( ["test.sh"], stdin=sp.PIPE)
    data = "".join([chr((s)%17) for s in range(0,33)])
    os.write(pipe.stdin.fileno(), data)
    pipe.stdin.write("endoffile")

if __name__ == "__main__":
    test()

它调用以下简单的bash shell脚本，该脚本仅将stdin写入文件(脚本称为test.sh)

It calls the following simple bash shell script which simply writes stdin to a file (script is called test.sh)

#!/bin/bash
VALUE=$(cat)

echo "$VALUE" >> /tmp/test.txt

当我运行python代码时，我希望test.txt包含两次0x01..0x10值，然后是字符串"endoffile"

When I run the python code I expect test.txt to contain the values 0x01..0x10 two times, and after that the string "endoffile"

但这是文件的十六进制转储:

However here's a hexdump of the file:

0000000: 0102 0304 0506 0708 090a 0b0c 0d0e 0f10  ................
0000010: 0102 0304 0506 0708 090a 0b0c 0d0e 0f65  ...............e
0000020: 6e64 6f66 6669 6c65 0a                   ndoffile.

似乎缺少一个字节(0x10).

It appears that a byte is missing (0x10).

我在这里想念什么?

---更新

将test()函数更改为:

Changing the test() function to:

def test():
    pipe = sp.Popen( ["test.sh"], stdin=sp.PIPE)
    data = "".join([chr((s)%16+1) for s in range(0,32)])
    os.write(pipe.stdin.fileno(), data)
    pipe.stdin.write("endoffile")

似乎可以解决这个问题. 似乎与将chr(0)发送到管道有关.

Seems to solve that. It seems to be related to having chr(0) sent to pipe.

推荐答案

range()在右侧是排他的.

range(0, 33)是[0, ..., 32]，可能是因为通过这种方式您可以range(0, len(sequence))而不会出现一字不漏的错误.

range(0, 33) is [0, ..., 32], probably because this way you can range(0, len(sequence)) without off-by-one errors.

自32 % 17 == 15 == 0x0f起，您期望的字节'\x10'永远不会成为列表的一部分.

Since 32 % 17 == 15 == 0x0f, the byte '\x10' you are expecting was never part of the list in the first place.

修改1: 输出中还缺少零字符'\x00'.如果使用VALUE=$(cat)，则cat的输出将由外壳处理.

Edit 1: Also missing from the output are the zero characters '\x00'. If you use VALUE=$(cat) the output of cat is subject to processing by the shell.

SingleUnix/POSIX在此问题上似乎保持沉默.但是很明显，您不能将'\0'作为shell变量的值(或相关名称)的一部分，因为Unix环境要求两者都为

SingleUnix/POSIX seems to be silent on the matter. It is however clear, that you cannot have '\0' as part of a shell variable's value (or name for that matter) since the Unix environment requires both to be C-style zero terminated strings. I actually would have expected the value of VALUE to be an empty string.

编辑2 经过一番挖掘，我可以说至少 ash实现会忽略'\0'处理反引号提供的输入.读取输入，直到EOF和空字符被显式跳过.

Edit 2 After some digging, I can say that at least the ash implementation ignores '\0' processing backtick-supplied input. Input is read until EOF and null characters are explicitly skipped.

bash相同，甚至具有显式(即使已注释掉)

bash does the same and even has an explicit (even if commented out) warning associated with the event.

这篇关于为什么在写入python子进程stdin管道时丢失数据?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！