如何在Python中进行并行编程? [英] How to do parallel programming in Python?

问题描述

对于C ++，我们可以使用OpenMP进行并行编程.但是，OpenMP不适用于Python.如果要并行执行python程序的某些部分，该怎么办?

For C++, we can use OpenMP to do parallel programming; however, OpenMP will not work for Python. What should I do if I want to parallel some parts of my python program?

代码的结构可以认为是:

The structure of the code may be considered as:

solve1(A)
solve2(B)

其中solve1和solve2是两个独立的功能.为了减少运行时间，如何并行而不是按顺序运行这种代码? 希望可以有人帮帮我.首先十分感谢. 代码是:

Where solve1 and solve2 are two independent function. How to run this kind of code in parallel instead of in sequence in order to reduce the running time? Hope someone can help me. Thanks very much in advance. The code is:

def solve(Q, G, n):
    i = 0
    tol = 10 ** -4

    while i < 1000:
        inneropt, partition, x = setinner(Q, G, n)
        outeropt = setouter(Q, G, n)

        if (outeropt - inneropt) / (1 + abs(outeropt) + abs(inneropt)) < tol:
            break

        node1 = partition[0]
        node2 = partition[1]

        G = updateGraph(G, node1, node2)

        if i == 999:
            print "Maximum iteration reaches"
    print inneropt

setinner和setouter是两个独立的功能.那是我要平行的地方...

Where setinner and setouter are two independent functions. That's where I want to parallel...

推荐答案

您可以使用多重处理模块.对于这种情况，我可以使用一个处理池:

You can use the multiprocessing module. For this case I might use a processing pool:

from multiprocessing import Pool
pool = Pool()
result1 = pool.apply_async(solve1, [A])    # evaluate "solve1(A)" asynchronously
result2 = pool.apply_async(solve2, [B])    # evaluate "solve2(B)" asynchronously
answer1 = result1.get(timeout=10)
answer2 = result2.get(timeout=10)

这将产生可以为您完成常规工作的进程.由于我们没有通过processes，因此它将为您计算机上的每个CPU内核生成一个进程.每个CPU内核可以同时执行一个进程.

This will spawn processes that can do generic work for you. Since we did not pass processes, it will spawn one process for each CPU core on your machine. Each CPU core can execute one process simultaneously.

如果要将列表映射到单个功能，请执行以下操作:

If you want to map a list to a single function you would do this:

args = [A, B]
results = pool.map(solve1, args)

不要使用线程，因为 GIL 会锁定对python对象的所有操作.

Don't use threads because the GIL locks any operations on python objects.

这篇关于如何在Python中进行并行编程?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！