在多个进程之间共享结果队列 [英] Sharing a result queue among several processes

问题描述

multiprocessing模块的文档显示了如何将队列传递给以multiprocessing.Process开始的进程.但是，如何与以apply_async开头的异步工作进程共享队列?我不需要动态加入或其他任何方式，而只是工人(反复)将其结果报告给基地的一种方式.

The documentation for the multiprocessing module shows how to pass a queue to a process started with multiprocessing.Process. But how can I share a queue with asynchronous worker processes started with apply_async? I don't need dynamic joining or anything else, just a way for the workers to (repeatedly) report their results back to base.

import multiprocessing
def worker(name, que):
    que.put("%d is done" % name)

if __name__ == '__main__':
    pool = multiprocessing.Pool(processes=3)
    q = multiprocessing.Queue()
    workers = pool.apply_async(worker, (33, q))

此操作失败，并显示: RuntimeError: Queue objects should only be shared between processes through inheritance. 我理解这意味着什么，并且我理解继承的建议，而不是要求进行酸洗/取消酸洗(以及所有Windows特殊限制).但是我如何以一种可行的方式通过队列?我找不到一个示例，并且我尝试了几种以各种方式失败的替代方法.请帮忙吗?

This fails with: RuntimeError: Queue objects should only be shared between processes through inheritance. I understand what this means, and I understand the advice to inherit rather than require pickling/unpickling (and all the special Windows restrictions). But how do I pass the queue in a way that works? I can't find an example, and I've tried several alternatives that failed in various ways. Help please?

推荐答案

尝试使用 multiprocessing.Manager 来管理您的队列，并使其他工作人员也可以访问该队列.

Try using multiprocessing.Manager to manage your queue and to also make it accessible to different workers.

import multiprocessing
def worker(name, que):
    que.put("%d is done" % name)

if __name__ == '__main__':
    pool = multiprocessing.Pool(processes=3)
    m = multiprocessing.Manager()
    q = m.Queue()
    workers = pool.apply_async(worker, (33, q))

这篇关于在多个进程之间共享结果队列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！