如何将Queue引用传递给pool.map_async()管理的函数? [英] How do you pass a Queue reference to a function managed by pool.map_async()?

问题描述

我希望一个运行时间较长的进程通过一个队列(或类似的东西)返回其进度，该队列将被馈送到进度栏对话框.该过程完成后，我还需要结果.此处的测试示例失败，并显示RuntimeError: Queue objects should only be shared between processes through inheritance.

I want a long-running process to return its progress over a Queue (or something similar) which I will feed to a progress bar dialog. I also need the result when the process is completed. A test example here fails with a RuntimeError: Queue objects should only be shared between processes through inheritance.

import multiprocessing, time

def task(args):
    count = args[0]
    queue = args[1]
    for i in xrange(count):
        queue.put("%d mississippi" % i)
    return "Done"

def main():
    q = multiprocessing.Queue()
    pool = multiprocessing.Pool()
    result = pool.map_async(task, [(x, q) for x in range(10)])
    time.sleep(1)
    while not q.empty():
        print q.get()
    print result.get()

if __name__ == "__main__":
    main()

我已经能够使用单个Process对象(允许我通过队列引用传递 am )来使它正常工作，但是我没有一个池来管理我的许多流程要启动.有什么建议可以使用更好的模式吗?

I've been able to get this to work using individual Process objects (where I am alowed to pass a Queue reference) but then I don't have a pool to manage the many processes I want to launch. Any advise on a better pattern for this?

推荐答案

以下代码似乎有效:

import multiprocessing, time

def task(args):
    count = args[0]
    queue = args[1]
    for i in xrange(count):
        queue.put("%d mississippi" % i)
    return "Done"


def main():
    manager = multiprocessing.Manager()
    q = manager.Queue()
    pool = multiprocessing.Pool()
    result = pool.map_async(task, [(x, q) for x in range(10)])
    time.sleep(1)
    while not q.empty():
        print q.get()
    print result.get()

if __name__ == "__main__":
    main()

请注意，队列是从manager.Queue()而不是multiprocessing.Queue()获得的.感谢Alex指出我朝这个方向.

Note that the Queue is got from a manager.Queue() rather than multiprocessing.Queue(). Thanks Alex for pointing me in this direction.

这篇关于如何将Queue引用传递给pool.map_async()管理的函数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！