将管道/连接作为上下文arg传递给多处理Pool.apply_async() [英] Passing a Pipe/Connection as context arg to multiprocessing Pool.apply_async()

问题描述

我想使用管道与池中的流程实例进行通信，但是出现错误:

I want to use pipes to talk to the process instances in my pool, but I'm getting an error:

让__p成为Pool()的实例:

Let __p be an instance of Pool():

    (master_pipe, worker_pipe) = Pipe()

    self.__p.apply_async(_worker_task, 
                         (handler_info, 
                          context_info,
                          worker_pipe))

执行此操作时，[显然，对于每个实例]都会出现以下错误:

When I execute this, I get the following error [for every instance, obviously]:

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/queues.py", line 376, in get
    task = get()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/queues.py", line 376, in get
TypeError: Required argument 'handle' (pos 1) not found
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 114, in run
    return recv()
    return recv()
    self._target(*self._args, **self._kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 102, in worker
TypeError: Required argument 'handle' (pos 1) not found
TypeError: Required argument 'handle' (pos 1) not found
    task = get()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/queues.py", line 376, in get
    return recv()
TypeError: Required argument 'handle' (pos 1) not found

该错误专门是指我尝试传递的Connection实例.如果我将其设置为无"，则工作人员会毫无错误地进行分叉.

The error is specifically referring to the Connection instance that I'm trying to pass. If I make it "None", the workers fork without error.

我不明白这一点，因为正如该文档通过示例强调的那样，我可以轻松地将相同的参数传递给Process()并使其完美运行:

I don't understand this since, as the document emphasizes through example, I can easily pass the same argument to a Process(), and have it work perfectly:

from multiprocessing import Pipe, Process
def call_me(p):
  print("Here: %s" % (p))

(master, worker) = Pipe()
p = Process(target=call_me, args=(worker,))
p.start()

Here: <read-write Connection, handle 6>

p.join()

推荐答案

这是一个 bug ，其中包含已在Python 3中修复.

This is a bug which has been fixed in Python 3.

最简单的解决方案是按照其他答案中的建议，通过队列的初始化程序传递队列.

Easiest solution is to pass the queue through the Pool's initializer as suggested in the other answer.

这篇关于将管道/连接作为上下文arg传递给多处理Pool.apply_async()的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！