我如何在 asyncio 中使用请求? [英] How could I use requests in asyncio?

问题描述

我想在 asyncio 中做并行 http 请求任务，但我发现 python-requests 会阻塞 asyncio 的事件循环.我找到了 aiohttp 但它无法提供使用 http 代理的 http 请求服务.

I want to do parallel http request tasks in asyncio, but I find that python-requests would block the event loop of asyncio. I've found aiohttp but it couldn't provide the service of http request using a http proxy.

所以我想知道是否有一种方法可以在 asyncio 的帮助下进行异步 http 请求.

So I want to know if there's a way to do asynchronous http requests with the help of asyncio.

推荐答案

要将请求(或任何其他阻塞库)与 asyncio 一起使用，您可以使用 BaseEventLoop.run_in_executor 在另一个线程中运行一个函数并从中产生结果以获得结果.例如:

To use requests (or any other blocking libraries) with asyncio, you can use BaseEventLoop.run_in_executor to run a function in another thread and yield from it to get the result. For example:

import asyncio
import requests

@asyncio.coroutine
def main():
    loop = asyncio.get_event_loop()
    future1 = loop.run_in_executor(None, requests.get, 'http://www.google.com')
    future2 = loop.run_in_executor(None, requests.get, 'http://www.google.co.uk')
    response1 = yield from future1
    response2 = yield from future2
    print(response1.text)
    print(response2.text)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

这将同时获得两个响应.

This will get both responses in parallel.

在 python 3.5 中，您可以使用新的 await/async 语法:

With python 3.5 you can use the new await/async syntax:

import asyncio
import requests

async def main():
    loop = asyncio.get_event_loop()
    future1 = loop.run_in_executor(None, requests.get, 'http://www.google.com')
    future2 = loop.run_in_executor(None, requests.get, 'http://www.google.co.uk')
    response1 = await future1
    response2 = await future2
    print(response1.text)
    print(response2.text)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

有关更多信息，请参阅 PEP0492.

See PEP0492 for more.

这篇关于我如何在 asyncio 中使用请求?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！