Python ThreadPoolExecutor 终止所有线程 [英] Python ThreadPoolExecutor terminate all threads

问题描述

我正在运行一段 python 代码，其中多个线程通过线程池执行程序运行.每个线程都应该执行一项任务(例如获取网页).我希望能够做的是终止所有线程，即使其中一个线程失败.例如:

I am running a piece of python code in which multiple threads are run through threadpool executor. Each thread is supposed to perform a task (fetch a webpage for example). What I want to be able to do is to terminate all threads, even if one of the threads fail. For instance:

with ThreadPoolExecutor(self._num_threads) as executor:
    jobs = []
    for path in paths:
        kw = {"path": path}
        jobs.append(executor.submit(start,**kw))
    for job in futures.as_completed(jobs):
        result = job.result()
        print(result)
def start(*args,**kwargs):
    #fetch the page
    if(success):
        return True
    else:
        #Signal all threads to stop

可以这样做吗?除非所有线程都成功，否则线程返回的结果对我来说是无用的，因此即使其中一个失败，我也想节省其余线程的一些执行时间并立即终止它们.实际代码显然正在执行相对冗长的任务，并有几个失败点.

Is it possible to do so? The results returned by threads are useless to me unless all of them are successful, so if even one of them fails, I would like to save some execution time of the rest of the threads and terminate them immediately. The actual code obviously is doing relatively lengthy tasks with a couple of failure points.

推荐答案

如果你已经完成了线程并想研究进程，那么这里的和平代码看起来非常有前途和简单，几乎与线程相同的语法，但是使用多处理模块.

If you are done with threads and want to look into processes, then this peace of code here looks very promising and simple, almost the same syntax as thread, but with the multiprocessing module.

当超时标志到期时进程终止，非常方便.

When the timeout flag expires the process is terminated, very convenient.

import multiprocessing

def get_page(*args, **kwargs):
    # your web page downloading code goes here

def start_get_page(timeout, *args, **kwargs):
    p = multiprocessing.Process(target=get_page, args=args, kwargs=kwargs)
    p.start()
    p.join(timeout)
    if p.is_alive():
        # stop the downloading 'thread'
        p.terminate()
        # and then do any post-error processing here

if __name__ == "__main__":
    start_get_page(timeout, *args, **kwargs)

这篇关于Python ThreadPoolExecutor 终止所有线程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！