傻瓜的Python多处理 [英] Python multiprocessing for dummies

问题描述

我试图找到一个简单的示例，清楚地显示单个任务被分割以进行多处理.

I'm trying to find a simple example that clearly shows a single task being divided for multi-processing.

坦率地说，许多示例过于复杂，因此流程很难使用.

Quite frankly, many of the examples are overly sophisticated thus making the flow tougher to play with.

有人愿意分享他们的突破性样本或例子吗?

Does anyone care to share their breakthrough sample or an example?

推荐答案

您的基本示例是这样:

>>> import multiprocessing as mp
>>> from math import sqrt
>>> worker_pool = mp.Pool()
>>> jobs = [0, 1, 4, 9, 16, 25] 
>>>
>>> # calculate jobs in blocking batch parallel
>>> results = worker_pool.map(sqrt, jobs)
>>> results
[0.0, 1.0, 2.0, 3.0, 4.0, 5.0]
>>>
>>> # calculate jobs in asynchronous parallel
>>> results = worker_pool.map_async(sqrt, jobs)
>>> results.get()
[0.0, 1.0, 2.0, 3.0, 4.0, 5.0]
>>>
>>> # calculate jobs in parallel with an unordered iterator
>>> results = worker_pool.imap_unordered(sqrt, jobs)
>>> list(results)  # NOTE: results may return out of order
[0.0, 1.0, 2.0, 3.0, 4.0, 5.0]
>>>
>>> # a single blocking job on another process
>>> worker_pool.apply(sqrt, [9])
3.0
>>> # a single asynchronous job on another process
>>> y = worker_pool.apply_async(sqrt, [9])
>>> y.get()
3.0
>>> # the same interface exists for threads
>>> thread_pool = mp.dummy.Pool()
>>> thread_pool.map(sqrt, jobs)
[0.0, 1.0, 2.0, 3.0, 4.0, 5.0]
>>>
>>> # finishing up, you should shut down your pools
>>> worker_pool.close()
>>> worker_pool.join()
>>> thread_pool.close()
>>> thread_pool.join()

如果您不希望批量并行处理，但想要更复杂的示例，示例可能会变得更加复杂.

Examples can get more complex if you don't want batch parallel, but want something more complicated.

这篇关于傻瓜的Python多处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！