在Numpy Python中使用多处理工具 [英] Utilizing the multi processing tool with Numpy Python

问题描述

我正在尝试使用具有numpy数组的多重处理工具.我想同时运行 func1()和 func2()，然后使用 np.concatenate()并放置 Solution_1 和 Solution_2 一起使用.我将如何做这样的事情?

I am trying to utilize multi processing tool with numpy arrays. I want to run func1() and func2() simultaneously and then use np.concatenate() and place Solution_1 and Solution_2 together. How will I be able to do such a thing?

模块:

import multiprocessing
import numpy as np

代码:

Numbers = np.array([1,2,3,4,5,6,7,8,9,10,11,12])
def func1():
     Solution_1 = Numbers + 10 
     return Solution_1 
def func2():
     Solution_2 = Numbers * 10
     return Solution_2 
#setting up the multi processing vars     
p1 = multiprocessing.Process(target=func1)
p2 = multiprocessing.Process(target=func2)

#running the multi processes 
if __name__ == "__main__":
    p1.start()
    p2.start()

推荐答案

例如:

import multiprocessing as mp
import numpy as np
num_cores = mp.cpu_count()

Numbers = np.array([1,2,3,4,5,6,7,8,9,10,11,12])
def func1():
     Solution_1 = Numbers + 10
     return Solution_1
def func2():
     Solution_2 = Numbers * 10
     return Solution_2

# Getting ready my cores, I left one aside
pool = mp.Pool(num_cores-1)
# This is to use all functions easily
functions = [func1, func2]
# This is to store the results
solutions = []
for function in functions:
    solutions.append(pool.apply(function, ()))

解决方案如下:

[array([11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22]),
 array([ 10,  20,  30,  40,  50,  60,  70,  80,  90, 100, 110, 120])]

您可以将其直接放在数据框中:

You could place that directly in a dataframe:

solutions = pd.DataFrame(solutions)
print(solutions)
    0   1   2   3   4   5   6   7   8    9   10   11
0  11  12  13  14  15  16  17  18  19   20   21   22
1  10  20  30  40  50  60  70  80  90  100  110  120

我在类似的情况下看到了/替代方案:

I see in a similar cases / alternatives:

• 全局变量，这与此处的原始案例非常相似: https://stackoverflow.com/a/40399562/7127519
• apply_async : https://stackoverflow.com/a/48933782/7127519
• 要交流的共享变量: https://stackoverflow.com/a/10415215/7127519

• Global Variables, this is very similar to the original case post here: https://stackoverflow.com/a/40399562/7127519
• apply_async: https://stackoverflow.com/a/48933782/7127519
• A shared variable to communicate: How can I recover the return value of a function passed to multiprocessing.Process?
• multiprocessing.Manager(): https://stackoverflow.com/a/10415215/7127519

还有其他人.

这篇关于在Numpy Python中使用多处理工具的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！