在python中并行执行任务 [英] Executing tasks in parallel in python

问题描述

我正在使用python 2.7，我有一些看起来像这样的代码:

I am using python 2.7, I have some code that looks like this:

task1()
task2()
task3()
dependent1()

task4()
task5()
task6()
dependent2()

dependent3()

这里唯一的依赖项如下:Dependent1需要等待任务1-3，Dependent2需要等待任务4-6，而Dependent3需要等待依赖项1-2 ...可以了:完整地运行首先并行执行6个任务，然后并行并行执行前两个依赖项.然后是最终依赖项

The only dependencies here are as follows: dependent1 needs to wait for tasks1-3, dependent2 needs to wait for tasks 4-6 and dependent3 needs to wait for dependents1-2... The following would be okay: running the whole 6 tasks first in parallel, then the first two dependents in parallel.. then the final dependent

我希望有尽可能多的任务并行运行，我在Google上搜索了一些模块，但是我希望避免使用外部库，并且不确定队列线程技术如何解决我的问题(也许有人可以推荐)很好的资源?)

I prefer to have as much tasks as possible running in parallel, I've googled for some modules but I was hoping to avoid external libraries, and not sure how the Queue-Thread technique can solve my problem (maybe someone can recommend a good resource?)

推荐答案

内置的 threading.Thread 类提供了您需要的所有内容:启动以启动新线程并加入以等待线程结束.

The builtin threading.Thread class offers all you need: start to start a new thread and join to wait for the end of a thread.

import threading

def task1():
    pass
def task2():
    pass
def task3():
    pass
def task4():
    pass
def task5():
    pass
def task6():
    pass

def dep1():
    t1 = threading.Thread(target=task1)
    t2 = threading.Thread(target=task2)
    t3 = threading.Thread(target=task3)

    t1.start()
    t2.start()
    t3.start()

    t1.join()
    t2.join()
    t3.join()

def  dep2():
    t4 = threading.Thread(target=task4)
    t5 = threading.Thread(target=task5)

    t4.start()
    t5.start()

    t4.join()
    t5.join()

def dep3():
    d1 = threading.Thread(target=dep1)
    d2 = threading.Thread(target=dep2)

    d1.start()
    d2.start()

    d1.join()
    d2.join()

d3 = threading.Thread(target=dep3)
d3.start()
d3.join()

您也可以使用 Queue.join 来加入，以等待线程结束

Alternatively to join you can use Queue.join to wait for the threads end.

这篇关于在python中并行执行任务的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！