使用POpen将变量发送到Stdin并将Stdout发送到变量 [英] Using POpen to send a variable to Stdin and to send Stdout to a variable

问题描述

在shell脚本中，我们具有以下命令:

In shell script, we have the following command:

/script1.pl < input_file| /script2.pl > output_file

我想使用模块subprocess在Python中复制以上流. input_file是一个大文件，我无法一次读取整个文件.因此，我想在管道流中传递每行input_string并返回字符串变量output_string，直到整个文件都流过为止.

I would like to replicate the above stream in Python using the module subprocess. input_file is a large file, and I can't read the whole file at once. As such I would like to pass each line, an input_string into the pipe stream and return a string variable output_string, until the whole file has been streamed through.

以下是第一次尝试:

process = subprocess.Popen(["/script1.pl | /script2.pl"], stdin = subprocess.PIPE, stdout = subprocess.PIPE, shell = True)
process.stdin.write(input_string)
output_string = process.communicate()[0]

但是，使用process.communicate()[0]会关闭流.我想使视频流保持打开状态以供将来的视频流使用.我尝试使用process.stdout.readline()代替，但是程序挂起.

However, using process.communicate()[0] closes the stream. I would like to keep the stream open for future streams. I have tried using process.stdout.readline(), instead, but the program hangs.

推荐答案

要在Python中使用subprocess模块模拟/script1.pl < input_file | /script2.pl > output_file shell命令:

To emulate /script1.pl < input_file | /script2.pl > output_file shell command using subprocess module in Python:

#!/usr/bin/env python
from subprocess import check_call

with open('input_file', 'rb') as input_file
    with open('output_file', 'wb') as output_file:
        check_call("/script1.pl | /script2.pl", shell=True,
                   stdin=input_file, stdout=output_file)

基于

您还可以使用 plumbum模块在Python中获取类似于shell的语法 :

You could also use plumbum module to get shell-like syntax in Python:

#!/usr/bin/env python
from plumbum import local

script1, script2 = local["/script1.pl"], local["/script2.pl"]
(script1 < "input_file" | script2 > "output_file")()

另请参见如何使用subprocess.Popen通过管道连接多个进程?

如果要逐行读取/写入，则答案取决于要运行的具体脚本.通常，如果您不小心，例如由于缓冲问题.<

If you want to read/write line by line then the answer depends on the concrete scripts that you want to run. In general it is easy to deadlock sending/receiving input/output if you are not careful e.g., due to buffering issues.

如果输入不取决于您的情况，那么可靠的跨平台方法是为每个流使用单独的线程:

If input doesn't depend on output in your case then a reliable cross-platform approach is to use a separate thread for each stream:

#!/usr/bin/env python
from subprocess import Popen, PIPE
from threading import Thread

def pump_input(pipe):
    try:
       for i in xrange(1000000000): # generate large input
           print >>pipe, i
    finally:
       pipe.close()

p = Popen("/script1.pl | /script2.pl", shell=True, stdin=PIPE, stdout=PIPE,
          bufsize=1)
Thread(target=pump_input, args=[p.stdin]).start()
try: # read output line by line as soon as the child flushes its stdout buffer
    for line in iter(p.stdout.readline, b''):
        print line.strip()[::-1] # print reversed lines
finally:
    p.stdout.close()
    p.wait()

这篇关于使用POpen将变量发送到Stdin并将Stdout发送到变量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！