我可以从multiprocessing.Process获取返回值吗? [英] Can I get a return value from multiprocessing.Process?

问题描述

我已经使用Python多处理模块在Monte Carlo代码中实现了一些简单的并行性.我的代码如下:

I've implemented some simple parallelism in a Monte Carlo code using the Python multiprocessing module. I have code that looks like:

montecarlos = [MonteCarlo(f,fargs) for fargs in farglist]
jobs = [multiprocessing.Process(mc) for mc in montecarlos]
for job in jobs: job.start()
for job in jobs: job.join()
results = [mc.results for mc in montecarlos]

但是，当我查看结果列表时，似乎蒙特卡洛迭代器尚未启动.我知道它们有，因为我可以让这些过程在蒙特卡洛步骤中打印出信息.所以我在做些愚蠢的事情.我以为job.join()会阻止结果列表被构建，直到一切运行完毕，因此mc.results字段将被更新.

However, when I look at the results list, it looks like the monte carlo iterators haven't even started. I know that they have, because I can have the processes print out information during the monte carlo steps. So I'm doing something dumb. I had thought the job.join() would keep the results list from being constructed until everything had run, and thus the mc.results field would be updated.

我意识到我还没有告诉您我的Monte Carlo例程的详细信息，希望这没有关系，并且我犯的错误是在我对多处理功能的解释中.预先感谢您提供的任何帮助.

I realize I haven't told you the details of my monte carlo routine, and hope that it doesn't matter, and that the mistake I'm making is in my interpretation of what multiprocessing does. Thanks in advance for any help you can offer.

推荐答案

已腌制MonteCarlo对象并将其发送给子进程以运行-由于本地mc从未运行过.

The MonteCarlo objects have been pickled and sent to child processes to be run - the .results attribute in this process isn't populated because the local mc has never been run.

如果您创建一个 multiprocessing.Queue ，则可以将其传递给每个MonteCarlo工作，完成后应将结果放入其中.然后，顶层可以等待队列中的值. (在引擎盖下，它会腌制和解开结果对象.)

If you create a multiprocessing.Queue, you can pass that into each MonteCarlo job, and when it finishes it should put the result in there. Then the top-level can wait for values from the queue. (Under the hood this will pickle and unpickle the result object.)

result_queue = multiprocessing.Queue()
montecarlos = [MonteCarlo(result_queue, f,fargs) for fargs in farglist]
jobs = [multiprocessing.Process(mc) for mc in montecarlos]
for job in jobs: job.start()
for job in jobs: job.join()
results = [result_queue.get() for mc in montecarlos]

这篇关于我可以从multiprocessing.Process获取返回值吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！