多重处理功能上的超时装饰器 [英] Time out decorator on a multprocessing function

问题描述

我直接从网上找到的一个例子中得到了这个装饰器:

I have this decorator taken directly from an example I found on the net:

class TimedOutExc(Exception):
    pass


def timeout(timeout):
    def decorate(f):
        def handler(signum, frame):
            raise TimedOutExc()

        def new_f(*args, **kwargs):

            old = signal.signal(signal.SIGALRM, handler)
            signal.alarm(timeout)

            try:
                result = f(*args, **kwargs)
            except TimedOutExc:
                return None
            finally:
                signal.signal(signal.SIGALRM, old)
            signal.alarm(0)
            return result

        new_f.func_name = f.func_name
        return new_f

    return decorate

如果f函数超时，它将引发异常.

It throws an exception if the f function times out.

嗯，它可以工作，但是当我在多处理功能上使用此装饰器并由于超时而停止时，它不会终止计算中涉及的进程.我该怎么办?

Well, it works but when I use this decorator on a multiprocessing function and stops due to a time out, it doesn't terminate the processes involved in the computation. How can I do that?

我不想启动异常并停止程序.基本上我想要的是f超时时，让它返回None，然后终止所涉及的进程.

I don't want to launch an exception and stop the program. Basically what I want is when f times out, have it return None and then terminate the processes involved.

推荐答案

虽然我同意亚伦回答的要点，但我还是要详细说明.

While I agree with the main point of Aaron's answer, I would like to elaborate a bit.

multiprocessing启动的进程必须在要装饰的函数中停止；我不认为可以通过装饰器本身来简单地完成此操作(装饰的函数是唯一知道它启动了哪些计算的实体).

The processes launched by multiprocessing must be stopped in the function to be decorated; I don't think that this can be done generally and simply from the decorator itself (the decorated function is the only entity that knows what calculations it launched).

您可以捕获自定义的TimedOutExc异常，而不是让修饰的函数捕获SIGALARM，这可能会更灵活.您的示例将变为:

Instead of having the decorated function catch SIGALARM, you can also catch your custom TimedOutExc exception–this might be more flexible. Your example would then become:

import signal
import functools

class TimedOutExc(Exception):
    """
    Raised when a timeout happens
    """

def timeout(timeout):
    """
    Return a decorator that raises a TimedOutExc exception
    after timeout seconds, if the decorated function did not return.
    """

    def decorate(f):

        def handler(signum, frame):
            raise TimedOutExc()

        @functools.wraps(f)  # Preserves the documentation, name, etc.
        def new_f(*args, **kwargs):

            old_handler = signal.signal(signal.SIGALRM, handler)
            signal.alarm(timeout)

            result = f(*args, **kwargs)  # f() always returns, in this scheme

            signal.signal(signal.SIGALRM, old_handler)  # Old signal handler is restored
            signal.alarm(0)  # Alarm removed

            return result

        return new_f

    return decorate

@timeout(10)
def function_that_takes_a_long_time():
    try:
        # ... long, parallel calculation ...
    except TimedOutExc:
        # ... Code that shuts down the processes ...
        # ...
        return None  # Or exception raised, which means that the calculation is not complete

这篇关于多重处理功能上的超时装饰器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！