如何并行启动函数，检查它们是否完成，并在python中启动一个新函数? [英] How to start functions in parallel, check if they are done, and start a new function in python?

问题描述

我想编写一个执行以下操作的python代码:

I want to write a python code that does the following:

• 首先，它会并行启动 3 个进程(或线程或其他).
• 然后在一个循环中，python 等待任何进程完成(并返回一些值)
• 然后，python代码启动一个新函数

最后，我希望 3 个进程始终并行运行，直到运行我需要运行的所有功能.这是一些伪代码:

In the end, I want 3 processes always running in parallel, until all functions I need to run are run. Here is some pseudocode:

import time
import random
from multiprocessing import Process

# some random function which can have different execution time
def foo():
    time.sleep(random.randint(10) + 2)
    return 42

# Start 3 functions
p = []
p.append(Process(target=foo))
p.append(Process(target=foo))
p.append(Process(target=foo))

while(True):
    
    # wait until one of the processes has finished
    ???

    # then add a new process so that always 3 are running in parallel
    p.append(Process(target=foo))

我很确定我想要什么还不清楚.请询问.

I am pretty sure it is not clear what I want. Please ask.

推荐答案

您真正想要的是启动三个进程并将您想要执行的作业提供给一个队列.然后只有三个进程，当一个进程完成时，它从队列中读取下一项并执行:

What you really want is to start three processes and feed a queue with jobs that you want executed. Then there will only ever be three processes and when one is finished, it reads the next item from the queue and executes that:

import time
import random
from multiprocessing import Process, Queue

# some random function which can have different execution time
def foo(a):
    print('foo', a)
    time.sleep(random.randint(1, 10) + 2)
    print(a)
    return 42

def readQueue(q):
    while True:
        item = q.get()
        if item:
            f,*args = item
            f(*args)
        else:
            return
    
if __name__ == '__main__':
    q = Queue()
    for a in range(4):  # create 4 jobs
        q.put((foo, a))
    for _ in range(3):  # sentinel for 3 processes
        q.put(None)

    # Start 3 processes
    p = []
    p.append(Process(target=readQueue, args=(q,)))
    p.append(Process(target=readQueue, args=(q,)))
    p.append(Process(target=readQueue, args=(q,)))

    for j in p:
        j.start()
    #time.sleep(10)
    for j in p:
        j.join()

这篇关于如何并行启动函数，检查它们是否完成，并在python中启动一个新函数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！