Python 多处理 - 独立处理字典中的每个键值对 [英] Python multiprocessing - Independently processing for each key-value pair in the dictionary

问题描述

我有一本看起来像这样的字典:

I have a dictionary that looks like this:

sampleData = {'x1': [1,2,3], 'x2': [4,5,6], 'x3': [7,8,9]}

我需要通过将数据传递给 blackBoxFunction 来对每个键和值对进行一些计算.这个函数需要时间来做处理.最终输出存储在单独的字典 finalValue = {} 中.

I need to do some calculation for each key and value pair by passing data to a blackBoxFunction. This function takes time to do the processing. The final output is stored in a separate dictionary finalValue = {}.

这是顺序执行的代码:

for key in sampleData.keys():
    finalValue[key] = []
    for i in range(0,len(sampleData[key])):
        for j in range(i,len(sampleData[key])):
            if(i!=j):
                finalValue[key].append(blackBoxFunction(sampleData[key][i],sampleData[key][j]))

但是，每个键值对的计算是相互独立的.

However, the calculation for each key and value pair is independent of each other.

我想知道如何在我的代码中使用多处理库来并行执行任务.

I want to know how can I use multiprocessing library in my code to perform the task in parallel.

最终的字典结构将类似于输入字典.

The final dictionary structure will look similar to the input dictionary.

finalValue 
{'x1': [31, 43, 53], 'x2': [97, 110, 131], 'x3': [135, 164, 137]}

推荐答案

尝试一些类似的东西

from multiprocessing import Pool

def pair_black_box(data):
    key, values = data
    res = []
    for i in range(0, len(values)):
        for j in range(i, len(values)):
            if(i != j):
               res.append(blackBoxFunction(values[i], values[j]))
    return key, res

p = Pool(3)
finalValue = dict(p.map(pair_black_box, sampleData.items()))

这篇关于Python 多处理 - 独立处理字典中的每个键值对的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！