multiprocessing.Pool:何时使用apply，apply_async或map? [英] multiprocessing.Pool: When to use apply, apply_async or map?

问题描述

我还没有看到带有 Pool.apply ， Pool.apply_async 和 Pool.map .我主要使用Pool.map;别人的优点是什么?

I have not seen clear examples with use-cases for Pool.apply, Pool.apply_async and Pool.map. I am mainly using Pool.map; what are the advantages of others?

推荐答案

在Python的早期，要使用任意参数调用函数，可以使用apply:

Back in the old days of Python, to call a function with arbitrary arguments, you would use apply:

apply(f,args,kwargs)

apply在Python2.7中仍然存在，尽管在Python3中不存在，并且通常不再使用.如今

apply still exists in Python2.7 though not in Python3, and is generally not used anymore. Nowadays,

f(*args,**kwargs)

是首选. multiprocessing.Pool模块尝试提供类似的界面.

is preferred. The multiprocessing.Pool modules tries to provide a similar interface.

Pool.apply类似于Python apply，不同之处在于函数调用是在单独的进程中执行的. Pool.apply阻止，直到功能完成.

Pool.apply is like Python apply, except that the function call is performed in a separate process. Pool.apply blocks until the function is completed.

Pool.apply_async也类似于Python的内置apply，不同之处在于调用立即返回而不是等待结果.返回AsyncResult对象.您调用其get()方法以检索函数调用的结果. get()方法将阻塞，直到功能完成.因此，pool.apply(func, args, kwargs)等同于pool.apply_async(func, args, kwargs).get().

Pool.apply_async is also like Python's built-in apply, except that the call returns immediately instead of waiting for the result. An AsyncResult object is returned. You call its get() method to retrieve the result of the function call. The get() method blocks until the function is completed. Thus, pool.apply(func, args, kwargs) is equivalent to pool.apply_async(func, args, kwargs).get().

与Pool.apply相比，Pool.apply_async方法还具有一个回调，如果提供该回调，则在函数完成时调用该回调.可以使用它代替调用get().

In contrast to Pool.apply, the Pool.apply_async method also has a callback which, if supplied, is called when the function is complete. This can be used instead of calling get().

例如:

import multiprocessing as mp
import time

def foo_pool(x):
    time.sleep(2)
    return x*x

result_list = []
def log_result(result):
    # This is called whenever foo_pool(i) returns a result.
    # result_list is modified only by the main process, not the pool workers.
    result_list.append(result)

def apply_async_with_callback():
    pool = mp.Pool()
    for i in range(10):
        pool.apply_async(foo_pool, args = (i, ), callback = log_result)
    pool.close()
    pool.join()
    print(result_list)

if __name__ == '__main__':
    apply_async_with_callback()

可能会产生诸如

[1, 0, 4, 9, 25, 16, 49, 36, 81, 64]

注意，与pool.map不同，结果的顺序可能与pool.apply_async调用的顺序不同.

Notice, unlike pool.map, the order of the results may not correspond to the order in which the pool.apply_async calls were made.

因此，如果您需要在一个单独的进程中运行一个函数，但是希望当前进程阻塞，直到该函数返回，请使用Pool.apply.像Pool.apply一样，Pool.map会阻塞直到返回完整的结果.

So, if you need to run a function in a separate process, but want the current process to block until that function returns, use Pool.apply. Like Pool.apply, Pool.map blocks until the complete result is returned.

如果您希望工作进程池异步执行许多功能调用，请使用Pool.apply_async.结果的顺序不能保证与Pool.apply_async的调用顺序相同.

If you want the Pool of worker processes to perform many function calls asynchronously, use Pool.apply_async. The order of the results is not guaranteed to be the same as the order of the calls to Pool.apply_async.

还请注意，您可以使用Pool.apply_async调用许多不同函数(并非所有调用都需要使用同一函数).

Notice also that you could call a number of different functions with Pool.apply_async (not all calls need to use the same function).

相反，Pool.map将相同的函数应用于许多参数. 但是，与Pool.apply_async不同，返回结果的顺序与参数的顺序相对应.

In contrast, Pool.map applies the same function to many arguments. However, unlike Pool.apply_async, the results are returned in an order corresponding to the order of the arguments.

这篇关于multiprocessing.Pool:何时使用apply，apply_async或map?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！