Python多处理速度 [英] Python multiprocessing speed

问题描述

我编写了这段代码以在计算机上测试Python的多处理:

I wrote this bit of code to test out Python's multiprocessing on my computer:

from multiprocessing import Pool

var = range(5000000)
def test_func(i):
    return i+1

if __name__ == '__main__':
    p = Pool()
    var = p.map(test_func, var)

我使用Unix的time命令对此进行了计时，结果是:

I timed this using Unix's time command and the results were:

real 0m2.914s
user 0m4.705s
sys  0m1.406s

然后，使用与我计时相同的var和test_func():

Then, using the same var and test_func() I timed:

var = map(test_func, var)

结果是

real 0m1.785s
user 0m1.548s
sys  0m0.214s

多处理代码是否应该比普通的map快得多?

Shouldn't the multiprocessing code be much faster than plain old map?

推荐答案

为什么要这样.

在map函数中，您只是按顺序调用该函数.

In map function, you are just calling the function sequentially.

多处理池创建一组将要映射您的任务的工作程序. 它正在协调多个工作进程来运行这些功能.

Multiprocessing pool creates a set of workers to which your task will be mapped. It is coordinating multiple worker processes to run these functions.

尝试在函数内部做一些重要的工作，然后为它们计时，看看多处理是否可以帮助您更快地进行计算.

Try doing some significant work inside your function and then time them and see if multiprocessing helps you to compute faster.

您必须了解使用多处理会产生开销.只有在计算工作量远远超过这些开销的情况下，您才能看到它的好处.

You have to understand that there will be overheads in using multiprocessing. Only when the computing effort is significantly greater than these overheads that you will see it's benefits.

请参阅Hellmann出色介绍中的最后一个示例: http://www.doughellmann. com/PyMOTW/multiprocessing/communication.html

See the last example in excellent introduction by Hellmann: http://www.doughellmann.com/PyMOTW/multiprocessing/communication.html

pool_size = multiprocessing.cpu_count() * 2
pool = multiprocessing.Pool(processes=pool_size,
                            initializer=start_process,
                            maxtasksperchild=2,
                            )
pool_outputs = pool.map(do_calculation, inputs)

您可以根据拥有的核心来创建池.

You create pools depending on cores that you have.

这篇关于Python多处理速度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！