异步等待函数中的 Python asyncio.semaphore [英] Python asyncio.semaphore in async-await function

问题描述

我正在尝试自学 Python 的异步功能.为此，我构建了一个异步网络抓取工具.我想限制我一次打开的连接总数，以便成为服务器上的好公民.我知道信号量是一个很好的解决方案，asyncio 库有一个 semaphore 内置类.我的问题是 Python 在 async 函数中使用 yield from 时会抱怨，因为您正在组合 yield 和 >await 语法.以下是我正在使用的确切语法...

I am trying to teach myself Python's async functionality. To do so I have built an async web scraper. I would like to limit the total number of connections I have open at once to be a good citizen on servers. I know that semaphore's are a good solution, and the asyncio library has a semaphore class built in. My issue is that Python complains when using yield from in an async function as you are combining yield and await syntax. Below is the exact syntax I am using...

import asyncio
import aiohttp

sema = asyncio.BoundedSemaphore(5)

async def get_page_text(url):
    with (yield from sema):
        try:
            resp = await aiohttp.request('GET', url)
            if resp.status == 200:
                ret_val = await resp.text()
        except:
            raise ValueError
        finally:
            await resp.release()
    return ret_val

引发此异常:

File "<ipython-input-3-9b9bdb963407>", line 14
    with (yield from sema):
         ^
SyntaxError: 'yield from' inside async function

我能想到的一些可能的解决方案...

Some possible solution I can think of...

1. 只需使用 @asyncio.coroutine 装饰器
2. 使用threading.Semaphore?这似乎可能会导致其他问题
3. 在 Python 3.6 的 Beta 版中为 this 试试这个 原因.

1. Just use the @asyncio.coroutine decorator
2. Use threading.Semaphore? This seems like it may cause other issues
3. Try this in the beta of Python 3.6 for this reason.

我对 Python 的异步功能非常陌生，所以我可能会遗漏一些明显的东西.

I am very new to Python's async functionality so I could be missing something obvious.

推荐答案

您可以使用 async with 语句来获取异步上下文管理器:

You can use the async with statement to get an asynchronous context manager:

#!/usr/local/bin/python3.5
import asyncio
from aiohttp import ClientSession


sema = asyncio.BoundedSemaphore(5)

async def hello(url):
    async with ClientSession() as session:
        async with sema, session.get(url) as response:
            response = await response.read()
            print(response)

loop = asyncio.get_event_loop()
loop.run_until_complete(hello("http://httpbin.org/headers"))

示例取自此处.该页面还是 asyncio 和 aiohttp 的入门指南.

Example taken from here. The page is also a good primer for asyncio and aiohttp in general.

这篇关于异步等待函数中的 Python asyncio.semaphore的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！