为什么 asyncio 的 run_in_executor 会阻止 tornado 的 get 处理程序? [英] Why asyncio's run_in_executor blocks tornado's get handler?
问题描述
我想在 tornado 的异步 GET 请求处理程序中运行一个缓慢的阻塞方法(实际上来自一个 3rd 方库).让方法只是:
I want to run a slow blocking method (actually from a 3rd-party library) in tornado's async GET request handler. Let the method be just:
def blocking_method(uid):
print("slow method started: ", uid)
time.sleep(10)
print("slow method done: ", uid)
return "slow method ({}) result".format(uid)
此外，我更喜欢在 asyncio 的事件循环中运行 Tornado 服务器:
Moreover, I prefer running the tornado server in asyncio's event loop:
if __name__ == '__main__':
tornado.platform.asyncio.AsyncIOMainLoop().install()
loop = asyncio.get_event_loop()
loop.run_until_complete(make_app())
loop.run_forever()
我知道 @run_in_executor 装饰器，但它不适合我，因为我使用 asyncio.要在异步协程中运行阻塞方法，我应该使用 asyncio.get_event_loop() 的 run_in_executor 方法.以下是如何做到这一点的示例，来自 this 答案:
I knew about @run_in_executor decorator but it's not suitable for me, since I use asyncio. To run a blocking method in async coroutine I should use run_in_executor method of asyncio.get_event_loop(). Here is an example how to do it, from this answer:
import asyncio
async def main():
loop = asyncio.get_event_loop()
executor = concurrent.futures.ThreadPoolExecutor(max_workers=4)
future1 = loop.run_in_executor(executor, blocking_method, 1)
future2 = loop.run_in_executor(executor, blocking_method, 2)
response1 = await future1
response2 = await future2
print(response1)
print(response2)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
它工作得很好，这是上一个脚本的输出:
And it works perfectly, here is output from the previous script:
slow method started: 1
slow method started: 2
slow method done: 2
slow method done: 1
slow method (1) result
slow method (2) result
但是如果我在 tornado 的 RequestHandler 的 async def get 方法中使用完全相同的技术:
But if I use the very same technique in async def get method of tornado's RequestHandler:
class AsyncHandler(tornado.web.RequestHandler):
async def get(self):
# simple counter to distinguish requests
self.application.counter += 1
in_msg = "Registered request #{}, working...".format(self.application.counter)
print(in_msg)
loop = asyncio.get_event_loop()
future = loop.run_in_executor(self.application.executor,
blocking_method,
self.application.counter)
result = await future
out_msg = "Request processed, result: {}".format(result)
print(out_msg)
self.write(out_msg)
它阻塞了处理程序的方法.换句话说，如果我在多个浏览器选项卡中打开 http://localhost:8888/(假设为两个)，那么我期望两个并行工作的请求，以下输出:
it blocks the method of the handler. In other words, If I open http://localhost:8888/ in several browser tabs (let it be two), then I expect two requests working in parallel, with the following output:
Registered request #1, working...
slow method started: 1
Registered request #2, working...
slow method started: 2
slow method done: 1
Request processed, result: slow method (1) result
slow method done: 2
Request processed, result: slow method (2) result
但是请求被执行结果:
Registered request #1, working...
slow method started: 1
slow method done: 1
Request processed, result: slow method (1) result
Registered request #2, working...
slow method started: 2
slow method done: 2
Request processed, result: slow method (2) result
那么，我错在哪里?我应该怎么做才能允许并行执行请求处理程序?
So, where am I wrong? What should I do to allow execution the request handler in parallel?
这是描述我的问题的完整脚本:
Here is full script that describes my problem:
import asyncio
import concurrent.futures
import time
import tornado.web
import tornado.platform
def blocking_method(uid):
print("slow method started: ", uid)
time.sleep(10)
print("slow method done: ", uid)
return "slow method ({}) result".format(uid)
class AsyncHandler(tornado.web.RequestHandler):
async def get(self):
# simple counter to distinguish requests
self.application.counter += 1
in_msg = "Registered request #{}, working...".format(self.application.counter)
print(in_msg)
loop = asyncio.get_event_loop()
future = loop.run_in_executor(self.application.executor,
blocking_method,
self.application.counter)
result = await future
out_msg = "Request processed, result: {}".format(result)
print(out_msg)
self.write(out_msg)
async def make_app():
handlers = [(r"/", AsyncHandler)]
app = tornado.web.Application(handlers, debug=True)
app.executor = concurrent.futures.ThreadPoolExecutor(max_workers=4)
app.counter = 0
app.listen(8888)
if __name__ == '__main__':
tornado.platform.asyncio.AsyncIOMainLoop().install()
loop = asyncio.get_event_loop()
loop.run_until_complete(make_app())
loop.run_forever()
推荐答案
浏览器会识别出您正试图在两个不同的选项卡中加载同一个页面，并延迟第二个请求，直到第一个请求完成.
Browsers will recognize that you are trying to load the same page in two different tabs and delay the second request until the first has finished.
- 向您的网址添加一些内容，使其独一无二.而不是 http://localhost:8888 在两个选项卡中，加载 http://localhost:8888/?x=1 和 http://localhost:8888/?x=2 在另一个中.
- 使用两种不同的浏览器.例如，即使在 Chrome 标签页中加载了相同的网址，Firefox 也能够加载该网址.
- Add something to your urls to make them unique. Instead of http://localhost:8888 in both tabs, load http://localhost:8888/?x=1 in one and http://localhost:8888/?x=2 in the other.
- Use two different browsers. For example, Firefox will be able to load a url even while that same url is being loaded in a Chrome tab.
这篇关于为什么 asyncio 的 run_in_executor 会阻止 tornado 的 get 处理程序?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
