如何使用Python多Pool.map填写numpy的阵列中的for循环 [英] How to use Python multiprocessing Pool.map to fill numpy array in a for loop

问题描述

我要填写for循环中的一个2D-numpy的数组，并使用多系计算。

I want to fill a 2D-numpy array within a for loop and fasten the calculation by using multiprocessing.

import numpy
from multiprocessing import Pool


array_2D = numpy.zeros((20,10))
pool = Pool(processes = 4)

def fill_array(start_val):
    return range(start_val,start_val+10)

list_start_vals = range(40,60)
for line in xrange(20):
    array_2D[line,:] = pool.map(fill_array,list_start_vals)
pool.close()

print array_2D

执行它的效果是，Python的运行4子过程并占据4个CPU核心但执行doesn't光洁度和不打印的阵列。如果我尝试写的阵列磁盘，什么都不会发生。

The effect of executing it is that Python runs 4 subprocesses and occupies 4 CPU cores BUT the execution doesn´t finish and the array is not printed. If I try to write the array to the disk, nothing happens.

谁能告诉我为什么吗？

推荐答案

下面的作品。首先，它是保护主块中的code的主要部分，以避免奇怪的副作用是个好主意。 poo.map的结果（）是包含每个价值评估中的迭代器 list_start_vals ，使得列表你没有之前创建 array_2D 。

The following works. First it is a good idea to protect the main part of your code inside a main block in order to avoid weird side effects. The result of poo.map() is a list containing the evaluations for each value in the iterator list_start_vals, such that you don't have to create array_2D before.

import numpy as np
from multiprocessing import Pool

def fill_array(start_val):
    return list(range(start_val, start_val+10))

if __name__=='__main__':
    pool = Pool(processes=4)
    list_start_vals = range(40, 60)
    array_2D = np.array(pool.map(fill_array, list_start_vals))
    pool.close() # ATTENTION HERE
    print array_2D

也许你会有麻烦使用 pool.close（），从@hpaulj你可以只是删除如果你有问题，这条线的评论...

perhaps you will have trouble using pool.close(), from the comments of @hpaulj you can just remove this line in case you have problems...

这篇关于如何使用Python多Pool.map填写numpy的阵列中的for循环的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！