多重处理功能上的超时装饰器 [英] Time out decorator on a multprocessing function
问题描述
我直接从网上找到的一个例子中得到了这个装饰器:
I have this decorator taken directly from an example I found on the net:
class TimedOutExc(Exception):
pass
def timeout(timeout):
def decorate(f):
def handler(signum, frame):
raise TimedOutExc()
def new_f(*args, **kwargs):
old = signal.signal(signal.SIGALRM, handler)
signal.alarm(timeout)
try:
result = f(*args, **kwargs)
except TimedOutExc:
return None
finally:
signal.signal(signal.SIGALRM, old)
signal.alarm(0)
return result
new_f.func_name = f.func_name
return new_f
return decorate
如果f函数超时,它将引发异常.
It throws an exception if the f function times out.
嗯,它可以工作,但是当我在多处理功能上使用此装饰器并由于超时而停止时,它不会终止计算中涉及的进程.我该怎么办?
Well, it works but when I use this decorator on a multiprocessing function and stops due to a time out, it doesn't terminate the processes involved in the computation. How can I do that?
我不想启动异常并停止程序.基本上我想要的是f超时时,让它返回None,然后终止所涉及的进程.
I don't want to launch an exception and stop the program. Basically what I want is when f times out, have it return None and then terminate the processes involved.
推荐答案
虽然我同意亚伦回答的要点,但我还是要详细说明.
While I agree with the main point of Aaron's answer, I would like to elaborate a bit.
multiprocessing
启动的进程必须在要装饰的函数中停止;我不认为可以通过装饰器本身来简单地完成此操作(装饰的函数是唯一知道它启动了哪些计算的实体).
The processes launched by multiprocessing
must be stopped in the function to be decorated; I don't think that this can be done generally and simply from the decorator itself (the decorated function is the only entity that knows what calculations it launched).
您可以捕获自定义的TimedOutExc
异常,而不是让修饰的函数捕获SIGALARM
,这可能会更灵活.您的示例将变为:
Instead of having the decorated function catch SIGALARM
, you can also catch your custom TimedOutExc
exception–this might be more flexible. Your example would then become:
import signal
import functools
class TimedOutExc(Exception):
"""
Raised when a timeout happens
"""
def timeout(timeout):
"""
Return a decorator that raises a TimedOutExc exception
after timeout seconds, if the decorated function did not return.
"""
def decorate(f):
def handler(signum, frame):
raise TimedOutExc()
@functools.wraps(f) # Preserves the documentation, name, etc.
def new_f(*args, **kwargs):
old_handler = signal.signal(signal.SIGALRM, handler)
signal.alarm(timeout)
result = f(*args, **kwargs) # f() always returns, in this scheme
signal.signal(signal.SIGALRM, old_handler) # Old signal handler is restored
signal.alarm(0) # Alarm removed
return result
return new_f
return decorate
@timeout(10)
def function_that_takes_a_long_time():
try:
# ... long, parallel calculation ...
except TimedOutExc:
# ... Code that shuts down the processes ...
# ...
return None # Or exception raised, which means that the calculation is not complete
这篇关于多重处理功能上的超时装饰器的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!