连接两个以 asyncio.subprocess.create_subprocess_exec() 启动的进程 [英] Connect two processes started with asyncio.subprocess.create_subprocess_exec()

问题描述

当使用旧式 subprocess.Popen() API 启动两个进程时，我可以轻松地将一个进程的 standard out 连接到 standard in 在另一个进程中，使用 | 连接命令时，以与 UNIX shell 相同的方式创建管道:

When starting two processes with the old school subprocess.Popen() API, I can easily connect standard out of one process to standard in of another process, creating a pipeline in the same way as a UNIX shell will do when connecting commands with |:

from subprocess import Popen, PIPE

process_1 = Popen(['ls'], stdout = PIPE)
process_2 = Popen(['wc'], stdin = process_1.stdout)

process_1.wait()
process_2.wait()

在使用来自 asyncio.subprocess.create_subprocess_exec()(或类似)的异步 API 时，我如何完成相同的操作?这是我试过的:

How can I accomplish the same when using the asynchronous API from asyncio.subprocess.create_subprocess_exec() (or similar)? This is what I tried:

from asyncio.events import get_event_loop
from asyncio.subprocess import PIPE, create_subprocess_exec

async def main():
    process_1 = await create_subprocess_exec('ls', stdout = PIPE)
    process_2 = await create_subprocess_exec('wc', stdin = process_1.stdout)

    await process_1.wait()
    await process_2.wait()

get_event_loop().run_until_complete(main())

但是对 create_subprocess_exec() 的第二次调用抱怨传递给 stdin 的参数没有 fileno(这是真的):

But the second call to create_subprocess_exec() complains that the argument passed to stdin has no fileno (which is true):

Traceback (most recent call last):
  File ".../test-async.py", line 11, in <module>
     get_event_loop().run_until_complete(main())
[...]
  File ".../test-async.py", line 6, in main
     process_2 = await create_subprocess_exec('wc', stdin = process_1.stdout)
[...]
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/subprocess.py", line 1388, in _get_handles
     p2cread = stdin.fileno()
AttributeError: 'StreamReader' object has no attribute 'fileno'

如何获得与上面同步示例相同的结果?

How can I get the same result as in the synchronous example above?

推荐答案

在 asyncio 中，process.stdout 实际上是一个 StreamReader，不是文件对象.可以通过process._transport._proc.stdout 访问文件对象.不幸的是，您将无法使用它，因为它已经在事件循环中注册以提供流接口 process.stdout.

In asyncio, process.stdout is actually a StreamReader, not a file object. The file object can be accessed through process._transport._proc.stdout. Unfortunately, you won't be able to use it since it has already been registered in the event loop in order to provide the stream interface process.stdout.

处理该问题的一种方法是创建自己的管道并将文件描述符传递给子进程:

One way to deal with the issue is to create your own pipe and pass the file descriptors to the subprocess:

async def main():
    read, write = os.pipe()
    process_1 = await create_subprocess_exec('ls', stdout=write)
    os.close(write)
    process_2 = await create_subprocess_exec('wc', stdin=read, stdout=PIPE)
    os.close(read)
    return await process_2.stdout.read()

请注意，write 文件描述符应在第一个子进程启动后显式关闭(除非您使用 subprocess.PIPE，否则它不会自动关闭).read 文件描述符也需要关闭，如 此处.

Note that the write file descriptor should be explicitly closed once the first subprocess is started (it is not automatically closed unless you use subprocess.PIPE). The read file descriptor also needs to be closed, as explained here.

这篇关于连接两个以 asyncio.subprocess.create_subprocess_exec() 启动的进程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！