如果我想给我的进程池更多的工作，我可以在 Pool.close() 之前调用 Pool.join() 吗? [英] If I want to give more work to my Process Pool, can I call Pool.join() before Pool.close()?

问题描述

multiprocessing 的文档 陈述关于Pool.join()的以下内容:

The documentation for multiprocessing states the following about Pool.join():

等待工作进程退出.在使用 join() 之前，必须先调用 close() 或 terminate().

Wait for the worker processes to exit. One must call close() or terminate() before using join().

我知道 Pool.close() 阻止任何其他任务被提交到池中；并且 Pool.join() 在继续父进程之前等待池完成.

I know that Pool.close() prevents any other task from being submitted to the pool; and that Pool.join() waits for the pool to finish before proceeding with the parent process.

那么，为什么我不能在 Pool.close() 之前调用 Pool.join() 在我想重用我的池来执行多个任务和然后最后 close() 很久以后呢?例如:

So, why can I not call Pool.join() before Pool.close() in the case when I want to reuse my pool for performing multiple tasks and then finally close() it much later? For example:

pool = Pool()
pool.map(do1)
pool.join() # need to wait here for synchronization
.
.
.
pool.map(do2)
pool.join() # need to wait here again for synchronization
.
.
.
pool.map(do3)
pool.join() # need to wait here again for synchronization
pool.close()

# program ends

为什么必须在使用 join() 之前调用 close() 或 terminate()"?

Why must one "call close() or terminate() before using join()"?

推荐答案

在你的情况下，你不需要在 map() 之后调用 join()，因为 map() 调用 blocks 直到所有结果都完成.但是你可以使用 map_async() 它不阻塞.

You need not call join() after map() in your case, because map() call blocks until all results are done. But you can use map_async() which does not blocking.

在close() 或terminate() 之前调用join() 是不正确的.因为 join() 是一个阻塞调用并等待工作进程退出.因此你不能在 join() 之后重用池.

Call join() before close() or terminate() is incorrect. Because join() is a blocking call and wait for the worker processes to exit. Therefore you can not reuse pool after join().

这篇关于如果我想给我的进程池更多的工作，我可以在 Pool.close() 之前调用 Pool.join() 吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！