multiprocessing.Pool:何时使用 apply、apply_async 或 map? [英] multiprocessing.Pool: When to use apply, apply_async or map?

问题描述

我还没有看到关于 Pool.apply, Pool.apply_async 和 Pool.map.我主要使用Pool.map；别人的优势是什么?

I have not seen clear examples with use-cases for Pool.apply, Pool.apply_async and Pool.map. I am mainly using Pool.map; what are the advantages of others?

推荐答案

回到 Python 的旧时代，要调用带有任意参数的函数，您可以使用 apply:

Back in the old days of Python, to call a function with arbitrary arguments, you would use apply:

apply(f,args,kwargs)

apply 在 Python2.7 中仍然存在，但在 Python3 中没有，一般不再使用.如今，

apply still exists in Python2.7 though not in Python3, and is generally not used anymore. Nowadays,

f(*args,**kwargs)

是首选.multiprocessing.Pool 模块试图提供类似的接口.

is preferred. The multiprocessing.Pool modules tries to provide a similar interface.

Pool.apply 与 Python apply 类似，只是函数调用是在单独的进程中执行的.Pool.apply 阻塞直到函数完成.

Pool.apply is like Python apply, except that the function call is performed in a separate process. Pool.apply blocks until the function is completed.

Pool.apply_async 也和 Python 内置的 apply 类似，只是调用会立即返回，而不是等待结果.返回一个 AsyncResult 对象.您调用它的 get() 方法来检索函数调用的结果.get() 方法会阻塞，直到函数完成.因此，pool.apply(func, args, kwargs) 等价于 pool.apply_async(func, args, kwargs).get().

Pool.apply_async is also like Python's built-in apply, except that the call returns immediately instead of waiting for the result. An AsyncResult object is returned. You call its get() method to retrieve the result of the function call. The get() method blocks until the function is completed. Thus, pool.apply(func, args, kwargs) is equivalent to pool.apply_async(func, args, kwargs).get().

与 Pool.apply 不同的是，Pool.apply_async 方法也有一个回调函数，如果提供了回调函数，就会在函数完成时调用.这可以用来代替调用 get().

In contrast to Pool.apply, the Pool.apply_async method also has a callback which, if supplied, is called when the function is complete. This can be used instead of calling get().

例如:

import multiprocessing as mp
import time

def foo_pool(x):
    time.sleep(2)
    return x*x

result_list = []
def log_result(result):
    # This is called whenever foo_pool(i) returns a result.
    # result_list is modified only by the main process, not the pool workers.
    result_list.append(result)

def apply_async_with_callback():
    pool = mp.Pool()
    for i in range(10):
        pool.apply_async(foo_pool, args = (i, ), callback = log_result)
    pool.close()
    pool.join()
    print(result_list)

if __name__ == '__main__':
    apply_async_with_callback()

可能会产生诸如

[1, 0, 4, 9, 25, 16, 49, 36, 81, 64]

请注意，与 pool.map 不同，结果的顺序可能与 pool.apply_async 调用的顺序不一致.

Notice, unlike pool.map, the order of the results may not correspond to the order in which the pool.apply_async calls were made.

因此，如果您需要在单独的进程中运行一个函数，但希望当前进程阻塞直到该函数返回，请使用Pool.apply.与Pool.apply 一样，Pool.map 会阻塞直到返回完整的结果.

So, if you need to run a function in a separate process, but want the current process to block until that function returns, use Pool.apply. Like Pool.apply, Pool.map blocks until the complete result is returned.

如果您希望工作进程池异步执行许多函数调用，请使用 Pool.apply_async.结果的顺序不能保证与Pool.apply_async的调用顺序相同.

If you want the Pool of worker processes to perform many function calls asynchronously, use Pool.apply_async. The order of the results is not guaranteed to be the same as the order of the calls to Pool.apply_async.

另请注意，您可以使用 Pool.apply_async 调用多个不同的函数(并非所有调用都需要使用相同的函数).

Notice also that you could call a number of different functions with Pool.apply_async (not all calls need to use the same function).

相反，Pool.map 对许多参数应用相同的函数.但是，与 Pool.apply_async 不同的是，返回结果的顺序与参数的顺序相对应.

In contrast, Pool.map applies the same function to many arguments. However, unlike Pool.apply_async, the results are returned in an order corresponding to the order of the arguments.

这篇关于multiprocessing.Pool:何时使用 apply、apply_async 或 map?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！