我可以从 multiprocessing.Process 中获得返回值吗? [英] Can I get a return value from multiprocessing.Process?

问题描述

我使用 Python 多处理模块在 Monte Carlo 代码中实现了一些简单的并行性.我的代码如下所示:

I've implemented some simple parallelism in a Monte Carlo code using the Python multiprocessing module. I have code that looks like:

montecarlos = [MonteCarlo(f,fargs) for fargs in farglist]
jobs = [multiprocessing.Process(mc) for mc in montecarlos]
for job in jobs: job.start()
for job in jobs: job.join()
results = [mc.results for mc in montecarlos]

但是，当我查看结果列表时，似乎蒙特卡罗迭代器甚至还没有启动.我知道他们有，因为我可以让流程在蒙特卡罗步骤中打印出信息.所以我在做一些愚蠢的事情.我原以为 job.join() 会阻止结果列表的构建，直到一切都运行完毕，因此 mc.results 字段将被更新.

However, when I look at the results list, it looks like the monte carlo iterators haven't even started. I know that they have, because I can have the processes print out information during the monte carlo steps. So I'm doing something dumb. I had thought the job.join() would keep the results list from being constructed until everything had run, and thus the mc.results field would be updated.

我意识到我没有告诉你我的 monte carlo 例程的详细信息，希望这没关系，我犯的错误在于我对多处理功能的解释.提前感谢您提供的任何帮助.

I realize I haven't told you the details of my monte carlo routine, and hope that it doesn't matter, and that the mistake I'm making is in my interpretation of what multiprocessing does. Thanks in advance for any help you can offer.

推荐答案

MonteCarlo 对象已被腌制并发送到要运行的子进程 - .results 属性在此过程中未填充，因为本地 mc 从未运行过.

The MonteCarlo objects have been pickled and sent to child processes to be run - the .results attribute in this process isn't populated because the local mc has never been run.

如果你创建一个multiprocessing.Queue，你可以将其传递给每个 MonteCarlo 作业，完成后它应该将结果放在那里.然后顶层可以等待队列中的值.(在引擎盖下，这将腌制和取消腌制结果对象.)

If you create a multiprocessing.Queue, you can pass that into each MonteCarlo job, and when it finishes it should put the result in there. Then the top-level can wait for values from the queue. (Under the hood this will pickle and unpickle the result object.)

result_queue = multiprocessing.Queue()
montecarlos = [MonteCarlo(result_queue, f,fargs) for fargs in farglist]
jobs = [multiprocessing.Process(mc) for mc in montecarlos]
for job in jobs: job.start()
for job in jobs: job.join()
results = [result_queue.get() for mc in montecarlos]

这篇关于我可以从 multiprocessing.Process 中获得返回值吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！