使用 Python 子进程将标准输出重定向到标准输入? [英] using Python subprocess to redirect stdout to stdin?

问题描述

我正在使用将二进制文件输出到 STDOUT 的子进程模块从 shell 调用程序.

我使用 Popen() 来调用程序，然后我想将流传递给 Python 包(称为pysam")中的一个函数，不幸的是它不能使用 Python 文件对象，但可以读取来自标准输入.所以我想做的是让 shell 命令的输出从 STDOUT 进入 STDIN.

如何在 Popen/subprocess 模块中完成此操作?这是我调用shell程序的方式:

p = subprocess.Popen(my_cmd, stdout=subprocess.PIPE, shell=True).stdout

这将读取my_cmd"的 STDOUT 输出并在 p 中获得一个流.由于我的 Python 模块无法直接从p"读取，因此我尝试使用以下方法将my_cmd"的 STDOUT 重定向回 STDIN:

p = subprocess.Popen(my_cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE, shell=True).stdout

然后我调用我的模块，它使用-"作为 STDIN 的占位符:

s = pysam.Samfile("-", "rb")

上面的调用只是意味着从 STDIN 读取(用-"表示)并将其作为二进制文件(rb")读取.

当我尝试这个时，我只是将二进制输出发送到屏幕，看起来 Samfile() 函数无法读取它.即使我删除了对 Samfile 的调用，也会发生这种情况，所以我认为是我对 Popen 的调用是问题所在，而不是下游步骤.

为了回应答案，我尝试了:

sys.stdin = subprocess.Popen(tagBam_cmd, stdout=subprocess.PIPE, shell=True).stdout打印打开 SAM.."s = pysam.Samfile("-","rb")打印完成?"sys.stdin = sys.__stdin__

这似乎挂了.我得到输出:

打开SAM..

但它永远不会超过 Samfile("-", "rb") 行.知道为什么吗?

知道如何解决这个问题吗?

编辑 2:我正在添加指向 Pysam 文档的链接，以防万一，我真的无法弄清楚.文档页面是:

http://wwwfgu.anat.ox.ac.uk/~andreas/documentation/samtools/usage.html

关于流的具体说明在这里:

http://wwwfgu.anat.ox.ac.uk/~andreas/documentation/samtools/usage.html#using-streams

特别是:

"""Pysam 不支持从真正的 python 文件对象读取和写入，但它支持从 stdin 和 stdout 读取和写入.以下示例从标准输入读取并写入标准输出:

infile = pysam.Samfile( "-", "r" )outfile = pysam.Samfile( "-", "w", template = infile )对于 infile 中的 s:outfile.write(s)

它也适用于 BAM 文件.以下脚本将标准输入上的 BAM 格式文件转换为标准输出上的 SAM 格式文件:

infile = pysam.Samfile( "-", "rb" )outfile = pysam.Samfile( "-", "w", template = infile )对于 infile 中的 s:outfile.write(s)

注意，只有文件打开方式需要从r改为rb."""

所以我只想获取来自 Popen 的流，它读取标准输出，并将其重定向到标准输入，这样我就可以使用 Samfile("-", "rb") ，因为上面的部分说明是可能的.

谢谢.

解决方案

如果你使用 stdout=subprocess.PIPE，你会在 stdout 上看到二进制，我有点困惑，但是，整体问题是如果你想欺骗 pysam 使用它，你需要使用 sys.stdin.

例如:

sys.stdin = subprocess.Popen(my_cmd, stdout=subprocess.PIPE, shell=True).stdouts = pysam.Samfile("-", "rb")sys.stdin = sys.__stdin__ # 恢复原来的标准输入

UPDATE:这假设 pysam 在 Python 解释器的上下文中运行，因此在指定-"时表示 Python 解释器的标准输入.不幸的是，它没有；当指定-"时，它直接从文件描述符 0 读取.

换句话说，它没有使用 Python 的 stdin (sys.stdin) 概念，因此替换它对 pysam.Samfile() 没有影响.也不可能从 Popen 调用中获取输出并以某种方式将其推送"到文件描述符 0；它是只读的，另一端连接到您的终端.

将输出放到文件描述符 0 上的唯一真正方法是将其移动到附加脚本中，然后从第一个脚本将两者连接在一起.这确保第一个脚本中 Popen 的输出最终会出现在第二个脚本的文件描述符 0 上.

因此，在这种情况下，您最好的选择是将其拆分为两个脚本.第一个将调用 my_cmd 并获取其输出并将其用作另一个调用 pysam.Samfile("-", "rb") 的 Python 脚本的第二个 Popen 的输入.

I'm making a call to a program from the shell using the subprocess module that outputs a binary file to STDOUT.

I use Popen() to call the program and then I want to pass the stream to a function in a Python package (called "pysam") that unfortunately cannot Python file objects, but can read from STDIN. So what I'd like to do is have the output of the shell command go from STDOUT into STDIN.

How can this be done from within Popen/subprocess module? This is the way I'm calling the shell program:

p = subprocess.Popen(my_cmd, stdout=subprocess.PIPE, shell=True).stdout

This will read "my_cmd"'s STDOUT output and get a stream to it in p. Since my Python module cannot read from "p" directly, I am trying to redirect STDOUT of "my_cmd" back into STDIN using:

p = subprocess.Popen(my_cmd, stdout=subprocess.PIPE, stdin=subprocess.PIPE, shell=True).stdout

I then call my module, which uses "-" as a placeholder for STDIN:

s = pysam.Samfile("-", "rb")

The above call just means read from STDIN (denoted "-") and read it as a binary file ("rb").

When I try this, I just get binary output sent to the screen, and it doesn't look like the Samfile() function can read it. This occurs even if I remove the call to Samfile, so I think it's my call to Popen that is the problem and not downstream steps.

EDIT: In response to answers, I tried:

sys.stdin = subprocess.Popen(tagBam_cmd, stdout=subprocess.PIPE, shell=True).stdout
print "Opening SAM.."                                                                                            
s = pysam.Samfile("-","rb")
print "Done?"
sys.stdin = sys.__stdin__    

This seems to hang. I get the output:

Opening SAM..

but it never gets past the Samfile("-", "rb") line. Any idea why?

Any idea how this can be fixed?

EDIT 2: I am adding a link to Pysam documentation in case it helps, I really cannot figure this out. The documentation page is:

http://wwwfgu.anat.ox.ac.uk/~andreas/documentation/samtools/usage.html

and the specific note about streams is here:

http://wwwfgu.anat.ox.ac.uk/~andreas/documentation/samtools/usage.html#using-streams

In particular:

""" Pysam does not support reading and writing from true python file objects, but it does support reading and writing from stdin and stdout. The following example reads from stdin and writes to stdout:

infile = pysam.Samfile( "-", "r" )
outfile = pysam.Samfile( "-", "w", template = infile )
for s in infile: outfile.write(s)

It will also work with BAM files. The following script converts a BAM formatted file on stdin to a SAM formatted file on stdout:

infile = pysam.Samfile( "-", "rb" )
outfile = pysam.Samfile( "-", "w", template = infile )
for s in infile: outfile.write(s)

Note, only the file open mode needs to changed from r to rb. """

So I simply want to take the stream coming from Popen, which reads stdout, and redirect that into stdin, so that I can use Samfile("-", "rb") as the above section states is possible.

thanks.

解决方案

I'm a little confused that you see binary on stdout if you are using stdout=subprocess.PIPE, however, the overall problem is that you need to work with sys.stdin if you want to trick pysam into using it.

For instance:

sys.stdin = subprocess.Popen(my_cmd, stdout=subprocess.PIPE, shell=True).stdout
s = pysam.Samfile("-", "rb")
sys.stdin = sys.__stdin__ # restore original stdin

UPDATE: This assumed that pysam is running in the context of the Python interpreter and thus means the Python interpreter's stdin when "-" is specified. Unfortunately, it doesn't; when "-" is specified it reads directly from file descriptor 0.

In other words, it is not using Python's concept of stdin (sys.stdin) so replacing it has no effect on pysam.Samfile(). It also is not possible to take the output from the Popen call and somehow "push" it on to file descriptor 0; it's readonly and the other end of that is connected to your terminal.

The only real way to get that output onto file descriptor 0 is to just move it to an additional script and connect the two together from the first. That ensures that the output from the Popen in the first script will end up on file descriptor 0 of the second one.

So, in this case, your best option is to split this into two scripts. The first one will invoke my_cmd and take the output of that and use it for the input to a second Popen of another Python script that invokes pysam.Samfile("-", "rb").

这篇关于使用 Python 子进程将标准输出重定向到标准输入?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！