并行处理python中的循环 [英] Parallelize for loop in python

问题描述

我有一个遗传算法，我想加快速度.我认为最简单的方法是通过pythons multiprocessing模块.在我的GA上运行cProfile之后，我发现大部分计算时间都发生在评估函数中.

I have a genetic algorithm which I would like to speed up. I'm thinking the easiest way to achieve this is by pythons multiprocessing module. After running cProfile on my GA, I found out that most of the computational time takes place in the evaluation function.

def evaluation():
    scores = []
    for chromosome in population:
        scores.append(costly_function(chromosome))

我将如何并行化此方法?重要的是，所有分数必须按照程序顺序运行的顺序附加.

How would I go about to parallelize this method? It is important that all the scores append in the same order as they would if the program would run sequentially.

我正在使用python 2.7

I'm using python 2.7

推荐答案

使用池(由于我在Google上看到了一些结果，因此我同时显示了imap和map，尽管我还没有看到证明，但可能无法订购map):

Use pool (I show both imap and map because of some results on google say map may not be OK for ordering though I have yet to see proof):

from multiprocessing import Pool
def evaluation(population):
    return list(Pool(processes=nprocs).imap(costly_function,population))

或(我用的是什么):

 return Pool(processes=nprocs).map(costly_function,population)

将nprocs定义为所需的并行进程数.

Define nprocs to the number of parallel process you want.

发件人: https://docs.python.org/dev/library/multiprocessing.html#multiprocessing.pool.Pool

这篇关于并行处理python中的循环的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！