当线程在 ThreadPoolExecutor() 中死亡时，我如何捕捉? [英] How can I catch when a thread dies in ThreadPoolExecutor()?

问题描述

我有一些非常简单的 python 代码，它通过 ThreadPoolExecutor() 通过各种进程运行一堆输入.现在，有时一个或多个线程会安静地消亡.其余线程继续运行并完成代码实际上很棒，但我想汇总某种类型的摘要，告诉我哪些线程已死(如果有).

I have some very simple python code that runs a bunch of inputs through various processes via ThreadPoolExecutor(). Now, sometimes one or more of the threads dies quietly. It is actually great that the rest of the threads continue on and the code completes, but I would like to put together some type of summary that tells me which, if any, of the threads have died.

我发现了几个例子，人们希望整个事情都关闭，但还没有看到任何进程继续进行的情况，并且在事后才报告发生错误的线程.

I've found several examples where folks want the whole thing to shut down, but haven't seen anything yet where the process continues on and the threads that have hit errors are just reported on after the fact.

非常感谢任何/所有想法！

Any/all thoughts greatly appreciated!

谢谢！

import concurrent.futures as cf

with cf.ThreadPoolExecutor() as executor:
     executor.map(process_a, process_a_inputs)
     executor.map(process_b, process_b_inputs)

推荐答案

Executor.map 不支持收集多个异常.但是，它的代码可以很容易地进行调整以返回发生故障的参数.

Executor.map does not support gathering more than one exception. However, its code can easily be adapted to return the arguments on which a failure occurred.

def attempt(executor: 'Executor', fn: 'Callable', *iterables):
    """Attempt to ``map(fn, *iterables)`` and return the args that caused a failure"""
    future_args = [(self.submit(fn, *args), args) for args in zip(*iterables)]

    def failure_iterator():
        future_args.reverse()
        while future_args:
            future, args = future_args.pop()
            try:
                future.result()
            except BaseException:
                del future
                yield args
    return failure_iterator()

这可用于同时将参数映射"到函数，然后检索任何失败.

This can be used to concurrently "map" arguments to functions, and later retrieve any failures.

import concurrent.futures as cf

with cf.ThreadPoolExecutor() as executor:
     a_failures = attempt(executor, process_a, process_a_inputs)
     b_failures = attempt(executor, process_b, process_b_inputs)
     for args in a_tries:
         print(f'failed to map {args} onto a')
     for args in b_tries:
         print(f'failed to map {args} onto b')

这篇关于当线程在 ThreadPoolExecutor() 中死亡时，我如何捕捉?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！