与Python的POPEN击风格进程替换 [英] Bash style process substitution with Python's Popen

问题描述

在bash，那么你可以将进程的输出重定向轻松到一个临时文件的描述符，它是所有的bash这样的自动处理的：

In Bash you can easily redirect the output of a process to a temporary file descriptor and it is all automagically handled by bash like this:

$ mydaemon --config-file <(echo "autostart: True \n daemonize: True")

或者是这样的：

$ wc -l <(ls)
15 /dev/fd/63

怎么看它是不是标准输入重定向：

see how it is not stdin redirection:

$ vim <(echo "Hello World") 
vim opens a text file containing "Hello world"
$ echo  "Hello World" | vim
Vim: Warning: Input is not from a terminal

您可以在第二个例子是如何的bash会自动创建一个文件描述符，并允许你的程序的输出传递到另一个程序看到的。

You can see in the second example how bash automatically creates a file descriptor and allows you to pass the output of a program to another program.

现在到我的问题：我如何可以做同样的事情与Python，子模块中使用POPEN

Now onto my question: How can I do the same thing with Python, using Popen in the subprocess module?

我一直在使用kmers的正常文件，只是读它，但我的程序现在可以生成基于用户参数在运行时kmers的具体名单。我想避免编写一个临时文件手动因为处理文件权限可能会导致问题，为我的原始用户。

I have been using a normal file of kmers and just reading it in, but my program now generates a specific list of kmers at runtime based on user parameters. I'd like to avoid writing to a temporary file manually because dealing with file permissions could cause problems for my primitive users.

下面是我的code运行我的程序和实际文件kmer_file

Here is my code to run my program and capture the stdout with an actual file "kmer_file"

input_file = Popen(["pram_axdnull", str(kmer), input_file, kmer_file], stdout=PIPE)

我创建了一个函数调用generate_kmers它返回一个可以写入到文件中轻松（包括新行）或一个StringIO的字符串。我也有一个python脚本是独立做同样的事情。

I created a function called generate_kmers which returns a string that can be written out to a file easily (includes newlines) or to a StringIO. I also have a python script that is standalone to do the same thing

所以，现在我想在把它作为我的第3个参数：

So now I want to pass it in as my 3rd parameter:

这不工作：

kmer_file = stringIO(generate_kmers(3))
input_file = Popen(["pram_axdnull", str(kmer), input_file, kmer_file], stdout=PIPE)

也做到这一点：

kmer_file = Popen(["generate_kmers", str(kmer)], stdout=PIPE)
input_file = Popen(["pram_axdnull", str(kmer), input_file, kmer_file.stdout], stdout=PIPE)

所以我的想法。

有谁知道解决这个好办法？我使用shell = true选项思维和使用＆LT的实际bashism;（），但我还没有想通了这一点。

Does anyone know of a good way to resolve this? I was thinking using the shell=True option and using the actual bashism of <() but I haven't figured that out.

感谢您！

推荐答案

如果 pram_axdnull 理解 - 会议意味着：从标准输入读取，那么你可以：

If pram_axdnull understands "-" convention to mean: "read from stdin" then you could:

p = Popen(["pram_axdnull", str(kmer), input_filename, "-"],
          stdin=PIPE, stdout=PIPE)
output = p.communicate(generate_kmers(3))[0]

如果由外部处理生成的输入：

If the input is generated by external process:

kmer_proc = Popen(["generate_kmers", str(kmer)], stdout=PIPE)
p = Popen(["pram_axdnull", str(kmer), input_filename, "-"],
          stdin=kmer_proc.stdout, stdout=PIPE)
kmer_proc.stdout.close()
output = p.communicate()[0]

如果 pram_axdnull 的不的理解 - 约定：

import os
import tempfile
from subprocess import check_output

with tempfile.NamedTemporaryFile() as file:
    file.write(generate_kmers(3))
    file.delete = False

try:
    p = Popen(["pram_axdnull", str(kmer), input_filename, file.name],
              stdout=PIPE)
    output = p.communicate()[0]
    # or
    # output = check_output(["pram_axdnull", str(kmer), input_filename, 
                             file.name])
finally:
    os.remove(file.name)

要使用外部程序生成的临时文件：

To generate temporary file using external process:

from subprocess import check_call

with tempfile.NamedTemporaryFile() as file:
    check_call(["generate_kmers", str(kmer)], stdout=file)
    file.delete = False

要避免等待中，即生成的所有kmers，写入/读取同时kmers，你可以使用 os.mkfifo（）在Unix（由@cdarke建议）

To avoid waiting for all kmers to be generated i.e., to write/read kmers simultaneously, you could use os.mkfifo() on Unix (suggested by @cdarke):

import os
import shutil
import tempfile
from contextlib import contextmanager
from subprocess import Popen, PIPE

@contextmanager
def named_pipe():
    dirname = tempfile.mkdtemp()
    try:
        path = os.path.join(dirname, 'named_pipe')
        os.mkfifo(path)
        yield path
    finally:
        shutil.rmtree(dirname)

with named_pipe() as path:
    p = Popen(["pram_axdnull", str(kmer), input_filename, path],
              stdout=PIPE) # read from path
    with open(path, 'wb') as wpipe:
        kmer_proc = Popen(["generate_kmers", str(kmer)],
                          stdout=wpipe) # write to path
    output = p.communicate()[0]
    kmer_proc.wait()

这篇关于与Python的POPEN击风格进程替换的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！