龙卷风阻止异步请求 [英] Tornado blocking asynchronous requests

问题描述

使用Tornado，我有一个Get请求，它花了很长时间，因为它向另一个Web服务发出了许多请求并处理了数据，可能要花几分钟才能完全完成.我不希望这阻止整个Web服务器响应当前的其他请求.

Using Tornado, I have a Get request that takes a long time as it makes many requests to another web service and processes the data, could take minutes to fully complete. I don't want this to block the entire web server from responding to other requests, which it currently does.

据我所知，Tornado是单线程的，并且同步执行每个请求，即使它异步处理它们(对此仍然感到困惑).漫长过程中的某些部分可能是暂停点，以允许服务器处理其他请求(可能的解决方案?).我在Heroku上仅由一个工作人员运行它，所以不确定如何转换为产生新线程或进行多处理，而我没有使用python的经验.

As I understand it, Tornado is single threaded and executes each request synchronously, even though it handles them asynchronously (still confused on that bit). There are parts of the long process that could be pause points to allow the server to handle other requests (possible solution?). I'm running it on Heroku with a single worker, so not sure how that translates into spawning a new thread or multiprocessing, which I have no experience in with python.

这是我要执行的操作:客户端进行get调用以启动该过程，然后每隔5秒循环遍历另一个get调用以检查状态并使用新信息更新页面(长时间轮询也会工作，但遇到了同样的问题).问题在于，启动长进程会阻止所有新的get请求(或新的长轮询会话)，直到完成为止.

Here is what I'm trying to do: the client makes the get call to start the process, then I loop through another get call every 5 seconds to check the status and update the page with new information (long polling would also work but running into the same issue). Problem is that starting the long process blocks all new get requests (or new long polling sessions) until it completes.

是否有一种简便的方法来启动此长获取电话，并且不会在此过程中阻塞整个Web服务器?我可以在代码中说些什么.暂停，先处理待处理的请求，然后继续"?

Is there an easy way to kick off this long get call and not have it block the entire web server in the process? Is there anything I can put in the code to say.. "pause, go handle pending requests then continue on"?

我需要在ProcessHandler上发起一个get请求.然后，我需要在ProcessHandler运行时继续能够查询StatusHandler.

I need to initiate a get request on ProcessHandler. I then need to continue to be able to query StatusHandler while ProcessHandler is running.

示例:

class StatusHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
       self.render("status.html")

class ProcessHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
       self.updateStatus("0")
       result1 = self.function1()
       self.updateStatus("1")
       result2 = self.function2(result1)
       self.updateStatus("2")
       result3 = self.function3(result2)
       self.updateStatus("3")
       self.finish()

推荐答案

这是一个完整的示例Tornado应用程序，它使用异步HTTP客户端和gen.Task模块使事情变得简单.

Here's a complete sample Tornado app that uses the Async HTTP client and the gen.Task module to make things simple.

如果您在文档中阅读了有关gen.Task的更多信息，您会看到实际上可以同时调度多个请求.这是使用Tornado的核心思想，即一切都不会阻塞并且仍保持单个进程.

If you read more about gen.Task in the docs you'll see that you can actually dispatch multiple requests at the same time. This is using the core idea of Tornado where everything is no blocking and still maintaining a single process.

更新:我添加了一个线程处理程序，以演示如何将工作分派到第二个线程中并在完成后接收callback().

Update: I've added a Thread handler to demonstrate how you could dispatch work into a second thread and receive the callback() when it's done.

import os
import threading
import tornado.options
import tornado.ioloop
import tornado.httpserver
import tornado.httpclient
import tornado.web
from tornado import gen
from tornado.web import asynchronous

tornado.options.define('port', type=int, default=9000, help='server port number (default: 9000)')
tornado.options.define('debug', type=bool, default=False, help='run in debug mode with autoreload (default: False)')

class Worker(threading.Thread):
   def __init__(self, callback=None, *args, **kwargs):
        super(Worker, self).__init__(*args, **kwargs)
        self.callback = callback

   def run(self):
        import time
        time.sleep(10)
        self.callback('DONE')

class Application(tornado.web.Application):
    def __init__(self):
        handlers = [
            (r"/", IndexHandler),
            (r"/thread", ThreadHandler),
        ]
        settings = dict(
            static_path = os.path.join(os.path.dirname(__file__), "static"),
            template_path = os.path.join(os.path.dirname(__file__), "templates"),
            debug = tornado.options.options.debug,
        )
        tornado.web.Application.__init__(self, handlers, **settings)

class IndexHandler(tornado.web.RequestHandler):
    client = tornado.httpclient.AsyncHTTPClient()

    @asynchronous
    @gen.engine
    def get(self):
        response = yield gen.Task(self.client.fetch, "http://google.com")

        self.finish("Google's homepage is %d bytes long" % len(response.body))

class ThreadHandler(tornado.web.RequestHandler):
    @asynchronous
    def get(self):
        Worker(self.worker_done).start()

    def worker_done(self, value):
        self.finish(value)

def main():
    tornado.options.parse_command_line()
    http_server = tornado.httpserver.HTTPServer(Application())
    http_server.listen(tornado.options.options.port)
    tornado.ioloop.IOLoop.instance().start()

if __name__ == "__main__":
    main()

这篇关于龙卷风阻止异步请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！