如何在 Python 中进行并行编程? [英] How to do parallel programming in Python?
问题描述
对于C++,我们可以使用OpenMP进行并行编程;但是,OpenMP 不适用于 Python.如果我想并行处理我的 Python 程序的某些部分,我该怎么办?
For C++, we can use OpenMP to do parallel programming; however, OpenMP will not work for Python. What should I do if I want to parallel some parts of my python program?
代码的结构可以认为是:
The structure of the code may be considered as:
solve1(A)
solve2(B)
其中solve1
和solve2
是两个独立的函数.如何并行而不是顺序运行这种代码以减少运行时间?代码是:
Where solve1
and solve2
are two independent function. How to run this kind of code in parallel instead of in sequence in order to reduce the running time?
The code is:
def solve(Q, G, n):
i = 0
tol = 10 ** -4
while i < 1000:
inneropt, partition, x = setinner(Q, G, n)
outeropt = setouter(Q, G, n)
if (outeropt - inneropt) / (1 + abs(outeropt) + abs(inneropt)) < tol:
break
node1 = partition[0]
node2 = partition[1]
G = updateGraph(G, node1, node2)
if i == 999:
print "Maximum iteration reaches"
print inneropt
其中setinner
和setouter
是两个独立的函数.这就是我想要并行的地方...
Where setinner
and setouter
are two independent functions. That's where I want to parallel...
推荐答案
您可以使用 multiprocessing模块.对于这种情况,我可能会使用处理池:
You can use the multiprocessing module. For this case I might use a processing pool:
from multiprocessing import Pool
pool = Pool()
result1 = pool.apply_async(solve1, [A]) # evaluate "solve1(A)" asynchronously
result2 = pool.apply_async(solve2, [B]) # evaluate "solve2(B)" asynchronously
answer1 = result1.get(timeout=10)
answer2 = result2.get(timeout=10)
这将产生可以为您完成通用工作的进程.由于我们没有通过 processes
,它会为您机器上的每个 CPU 内核生成一个进程.每个 CPU 内核可以同时执行一个进程.
This will spawn processes that can do generic work for you. Since we did not pass processes
, it will spawn one process for each CPU core on your machine. Each CPU core can execute one process simultaneously.
如果要将列表映射到单个函数,可以这样做:
If you want to map a list to a single function you would do this:
args = [A, B]
results = pool.map(solve1, args)
不要使用线程,因为 GIL 会锁定对 Python 对象的任何操作.
Don't use threads because the GIL locks any operations on python objects.
这篇关于如何在 Python 中进行并行编程?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!