在Tornado中的线程中运行长块函数时遇到问题 [英] Having problems running a long blocking function in a thread in Tornado

问题描述

我对龙卷风很陌生.只是看到如何处理在龙卷风中阻塞的请求.我在单独的线程中运行阻塞代码.但是，主线程仍会阻塞，直到线程功能完成为止.我不是在这里使用gen.coroutine，但是已经尝试过，结果是一样的

I am very new to Tornado. Was just seeing how to handle a request which blocks in Tornado. I run the blocking code in a separate thread. However the main thread still blocks till the threaded function finishes. I am not using gen.coroutine here, but have tried that and the result is the same

counter = 0

def run_async(func):
    @wraps(func)
    def function_in_a_thread(*args, **kwargs):
        func_t = Thread(target=func, args=args, kwargs=kwargs)
        func_t.start()
    return function_in_a_thread

def long_blocking_function(index, sleep_time, callback):
    print "Entering run counter:%s" % (index,)
    time.sleep(sleep_time)
    print "Exiting run counter:%s" % (index,)

    callback('keyy' + index)


class FooHandler(tornado.web.RequestHandler):

    @web.asynchronous
    def get(self):

        global counter
        counter += 1
        current_counter = str(counter)

        print "ABOUT to spawn thread for counter:%s" % (current_counter,)
        long_blocking_function(
            index=current_counter,
            sleep_time=5, callback=self.done_waiting)


        print "DONE with the long function"

    def done_waiting(self, response):
        self.write("Whatever %s " % (response,))
        self.finish()


class Application(tornado.web.Application):
    def __init__(self):
        handlers = [(r"/foo", FooHandler),
                    ]

        settings = dict(
            debug=True,
        )

        tornado.web.Application.__init__(self, handlers, **settings)


def main():
    application = Application()
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

if __name__ == "__main__":
    main()

当我背对背发出请求时，FooHandler块将停止，直到long_blocking_function完成才接收到任何请求.所以我最终看到类似的东西

When I issue back to back requests the FooHandler blocks and does not recieve any requests till the long_blocking_function finishes. So I end up seeing something like

ABOUT to spawn thread for counter:1
Entering run counter:1
Exiting run counter:1
DONE with the long function
ABOUT to spawn thread for counter:2
Entering run counter:2
Exiting run counter:2
DONE with the long function
ABOUT to spawn thread for counter:3
Entering run counter:3
Exiting run counter:3
DONE with the long function

我一直期待这些事情(因为我在对long_blocking_function的第一次调用完成之前发出了多个请求)，但是只看到了与上述类似的痕迹

I was expecting something along these lines(as I am issuing multiple requests before the first call to long_blocking_function finishes) but am only seeing trace similar to above

ABOUT to spawn thread for counter:1
DONE with the long function
ABOUT to spawn thread for counter:2
DONE with the long function
ABOUT to spawn thread for counter:3
DONE with the long function
ABOUT to spawn thread for counter:4
DONE with the long function

我查看了 Tornado阻止异步请求，并尝试了两种解决方案.但是，当我对同一处理程序执行背对背的请求时，它们都处于阻塞状态.有人可以弄清楚我在做什么错吗?我知道龙卷风在多线程方面做得不好，但是我应该能够以非阻塞的方式从中运行新线程.

I have looked at Tornado blocking asynchronous requests and tried both the solutions. But both of them are blocking when I run them with back to back requests to the same handler. Can somebody figure out what I am doing wrong? I know tornado doesn't do well with multithreading, but I should be able to run a new thread from it, in a non-blocking way.

推荐答案

Tornado与 Python 2.x反向端口可用) ，因此您可以使用 ThreadPoolExecutor .

Tornado plays well with the concurrent.futures library (there's a Python 2.x backport available), so you could hand off your long running requests to a thread pool by using a ThreadPoolExecutor.

这项技术效果很好-我们使用它来处理长时间运行的数据库操作.当然，在现实世界中，您还希望以健壮和优美的方式处理超时和其他异常，但是我希望这个示例足以说明这个想法.

This technique works pretty well - we use it to handle long-running database operations. Of course, in the real world you'd also want to handle timeouts and other exceptions in a robust and graceful manner, but I hope this example is enough to illustrate the idea.

def long_blocking_function(index, sleep_time, callback):
    print ("Entering run counter:%s" % (index,))
    time.sleep(sleep_time)
    print ("Exiting run counter:%s" % (index,))
    return "Result from %d" % index


class FooHandler(tornado.web.RequestHandler):
    @tornado.gen.coroutine
    def get(self):
        global counter
        counter += 1
        current_counter = str(counter)

        print ("ABOUT to spawn thread for counter:%s" % (current_counter,))
        result = yield self.executor.submit(long_blocking_function,
                                            index=current_counter,
                                            sleep_time=5)
        self.write(result)
        print ("DONE with the long function")

这篇关于在Tornado中的线程中运行长块函数时遇到问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！