并行执行功能列表 [英] Parallel execution of a list of functions

问题描述

因此，使用多进程模块可以很容易地与不同的参数并行运行一个函数，如下所示:

So using the multiprocess module it is easy to run a function in parallel with different arguments like this:

from multiprocessing import Pool

def f(x):
    return x**2


p = Pool(2)
print(p.map(f, [1, 2]))

但是我有兴趣在同一参数上执行函数列表.假设我具有以下两个功能:

But I'm interested in executing a list of functions on the same argument. Suppose I have the following two functions:

def f(x):
    return x**2


def g(x):
    return x**3 + 2

如何为相同的参数(例如x = 1)并行执行它们?

How can I execute them in parallel for the same argument (e.g. x=1)?

推荐答案

您可以为此使用Pool.apply_async().您以(函数，argument_tuple)的形式捆绑任务，并将每个任务馈送到apply_async().

You can use Pool.apply_async() for that. You bundle up tasks in the form of (function, argument_tuple) and feed every task to apply_async().

from multiprocessing import Pool
from itertools import repeat


def f(x):
    for _ in range(int(50e6)): # dummy computation
        pass
    return x ** 2


def g(x):
    for _ in range(int(50e6)): # dummy computation
        pass
    return x ** 3


def parallelize(n_workers, functions, arguments):
    # if you need this multiple times, instantiate the pool outside and
    # pass it in as dependency to spare recreation all over again
    with Pool(n_workers) as pool:
        tasks = zip(functions, repeat(arguments))
        futures = [pool.apply_async(*t) for t in tasks]
        results = [fut.get() for fut in futures]
    return results


if __name__ == '__main__':

    N_WORKERS = 2

    functions = f, g
    results = parallelize(N_WORKERS, functions, arguments=(10,))
    print(results)

示例输出:

[100, 1000]

Process finished with exit code 0

这篇关于并行执行功能列表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！