multiprocessing.Pool.apply和multiprocessing.Pool.apply_async的目的 [英] Purpose of multiprocessing.Pool.apply and multiprocessing.Pool.apply_async

问题描述

请参见下面的示例和执行结果:

See example and execution result below:

#!/usr/bin/env python3.4
from multiprocessing import Pool
import time
import os

def initializer():
    print("In initializer pid is {} ppid is {}".format(os.getpid(),os.getppid()))

def f(x):
    print("In f pid is {} ppid is {}".format(os.getpid(),os.getppid()))
    return x*x

if __name__ == '__main__':
    print("In main pid is {} ppid is {}".format(os.getpid(), os.getppid()))
    with Pool(processes=4, initializer=initializer) as pool:  # start 4 worker processes
        result = pool.apply(f, (10,)) # evaluate "f(10)" in a single process
        print(result)

        #result = pool.apply_async(f, (10,)) # evaluate "f(10)" in a single process
        #print(result.get())

赠予:

$ ./pooleg.py
In main pid is 22783 ppid is 19542
In initializer pid is 22784 ppid is 22783
In initializer pid is 22785 ppid is 22783
In initializer pid is 22787 ppid is 22783
In f pid is 22784 ppid is 22783
In initializer pid is 22786 ppid is 22783
100

从输出中可以清楚地看到:创建了4个进程，但实际上只有其中一个完成了工作(称为f).

As is clear from the output: 4 processes were created but only one of them actually did the work (called f).

问题:为什么当工作f仅由一个进程完成时，为什么我要创建一个由1个以上的工作人员组成的池并调用apply()?对于apply_async()也是一样，因为在这种情况下，工作也只能由一名工人完成.

Question: Why would I create a pool of > 1 workers and call apply() when the work f is done only by one process ? And same thing for apply_async() because in that case also the work is only done by one worker.

我不了解这些功能有用的用例.

I don't understand the use cases in which these functions are useful.

推荐答案

首先，两者都旨在对自变量元组(单个函数调用)进行操作，与对可迭代对象进行操作的Pool.map变体相反.因此，当您仅观察到一次调用这些函数所使用的一个进程时，这并不是一个错误.

First off, both are meant to operate on argument-tuples (single function calls), contrary to the Pool.map variants which operate on iterables. So it's not an error when you observe only one process used when you call these functions only once.

您将使用Pool.apply_async而不是Pool.map版本之一，在该版本中，您需要对要分发的单个任务进行更精细的控制.

You would use Pool.apply_async instead of one of the Pool.map versions, where you need more fine grained control over the single tasks you want to distribute.

Pool.map版本采用可迭代的方式并将其分块到任务中，其中每个任务都具有 same (映射)目标功能. Pool.apply_async通常不会在具有1个以上工人的池中仅被调用一次.由于它是异步的，因此您可以遍历手动预捆绑的任务并将其提交给多个任务. 工人进程中的任何一个都尚未完成.您的任务列表可以包含不同的目标功能，就像您在此处中看到的那样.它还允许注册结果和错误的回调，例如此示例.

The Pool.map versions take an iterable and chunk them into tasks, where every task has the same (mapped) target function. Pool.apply_async typically isn't called only once with a pool of >1 workers. Since it's asynchronous, you can iterate over manually pre-bundled tasks and submit them to several worker-processes before any of them has completed. Your task-list here can consist of different target functions like you can see in this answer here. It also allows registering callbacks for results and errors like in this example.

这些属性使Pool.apply_async非常通用，并且是

These properties make Pool.apply_async pretty versatile and a first-choice tool for unusual problem scenarios you cannot get done with one of the Pool.map versions.

Pool.apply乍一看(和第二眼)并没有广泛使用.如果先用apply_async启动多个任务，然后又要完成一个任务，然后再用apply_async启动另一轮任务，则可以使用它来同步控制流.

Pool.apply indeed is not widely usefull at first sight (and second). You could use it to synchronize control flow in a scenario where you start up multiple tasks with apply_async first and then have a task which has to be completed before you fire up another round of tasks with apply_async.

使用Pool.apply可能还意味着当您已经有一个正在闲置的池时，让您不必为中间任务创建一个额外的流程.

Using Pool.apply could also just mean sparing you to create a single extra Process for an in-between task, when you already have a pool which is currently idling.

这篇关于multiprocessing.Pool.apply和multiprocessing.Pool.apply_async的目的的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！