限制Python中的函数执行 [英] Limit function execution in Python

问题描述

有很多类似的问题和答案，但我仍然找不到可靠的答案.

There are a lot of similar questions and asnwers, but I still can't find reliable answer.

所以，我有一个函数，可能运行太久.函数是私有的，从某种意义上说我无法更改代码.

So, I have a function, that can possibly run too long. Function is private, in sense that I can not change it code.

我想将其执行时间限制为60秒. 我尝试了以下方法:

I want to restrict its execution time to 60 seconds. I tried following approaches:

1. Python信号.不要在Windows和多线程加密(mod_wsgi)中使用.
2. 线程.不错的方法，但是线程无法停止，因此即使提高TimeoutException后它也仍然存在.
3. multiprocessing python模块.我在酸洗方面有问题，我不知道如何解决.我想做time_limit装饰器，并且在顶层导入所需的函数时出现问题.长时间执行的函数是实例方法，并且将其包装也无济于事...

1. Python signals. Don't work on Windows and in multithreaded envorinment (mod_wsgi).
2. Threads. Nice way, but thread can not be stopped, so that it lives even after raising TimeoutException.
3. multiprocessing python module. I have problems with pickling and I don't know how to solve them. I want to make time_limit decorator and there are problems with importing required function in top-level. Function that is executed long is instance method, and wrapping it also doesn't help...

那么，上述问题是否有好的解决方案? 我如何开始杀死线程? 如何使用子流程并避免酸洗问题? subprocess模块有帮助吗?

So, are there good solutions to the above problem? How to kill thread, that I started? How to use subprocesses and avoid problems with pickling? Is subprocess module of any help?

谢谢.

推荐答案

我认为多处理方法是您唯一的选择.您是正确的，线程不能被杀死(很好)，并且信号具有跨平台问题.这是一种多处理实现:

I think the multiprocessing approach is your only real option. You're correct that threads can't be killed (nicely) and signals have cross-platform issues. Here is one multiprocessing implementation:

import multiprocessing
import Queue

def timed_function(return_queue):
    do_other_stuff()
    return_queue.put(True)
    return

def main():

    return_queue = multiprocessing.Manager().Queue()

    proc = multiprocessing.Process(target=timed_function, args=(return_queue,))
    proc.start()

    try:

        # wait for 60 seconds for the function to return a value
        return_queue.get(timeout=60)

    except Queue.Empty:
        # timeout expired
        proc.terminate() # kill the subprocess
        # other cleanup

我知道您说过您遇到了酸洗问题，但是这些问题几乎总是可以通过重构解决.例如，您说您的long函数是一个实例方法.您可以 包装这些类型的函数以将其用于多处理:

I know you said that you have pickling issues, but those can almost always be resolved with refactoring. For example, you said that your long function is an instance method. You can wrap those kinds of functions to use them with multiprocessing:

class TestClass(object):
    def timed_method(self, return_queue):
        do_other_stuff()
        return_queue.put(True)
        return

要在工作人员池中使用该方法，请将此包装器添加到模块的顶层:

To use that method in a pool of workers, add this wrapper to the top-level of the module:

def _timed_method_wrapper(TestClass_object, return_queue):
    return TestClass_object(return_queue)

例如，现在您可以在同一类的其他方法上的类方法上使用apply_async:

Now you can, for example, use apply_async on this class method from a different method of the same class:

def run_timed_method():
    return_queue = multiprocessing.Manager().Queue()
    pool = multiprocessing.Pool()
    result = pool.apply_async(_timed_method_wrapper, args=(self, return_queue))

我很确定只有在使用 multiprocessing.Pool 而不是使用 multiprocessing.Process 对象.另外，我敢打赌，很多人会不赞成这种构造，因为您破坏了类提供的美观，清晰的抽象，并且还在类和其他随机包装函数之间建立了依赖关系.您必须要决定让代码更丑陋是否值得.

I'm pretty sure that these wrappers are only necessary if you're using a multiprocessing.Pool instead of launching the subprocess with a multiprocessing.Process object. Also, I bet a lot of people would frown on this construct because you're breaking the nice, clean abstraction that classes provide, and also creating a dependency between the class and this other random wrapper function hanging around. You'll have to be the one to decide if making your code more ugly is worth it or not.

这篇关于限制Python中的函数执行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！