异步和等待关键字到底是如何工作的?等待链的末尾是什么? [英] How do the async and await keywords work, exactly? What's at the end of the await chain?

问题描述

我有此代码:

async def foo(x):
    yield x
    yield x + 1

async def intermediary(y):
    await foo(y)

def bar():
    c = intermediary(5)

我应该怎样放置才能从c中取出5和6?

What do I put in bar to get the 5 and the 6 out of c?

我问是因为asyncio库似乎很神奇.而且我想确切地知道魔术是如何工作的.

I'm asking because the asyncio library seems like a lot of magic. And I want to know exactly how the magic works.

也许我想编写自己的调用read或write的函数，然后通知某个顶级循环，我写了它们正在等待文件描述符变为可读或可写的状态.

Maybe I want to write my own functions that call read or write and then inform some top level loop that I wrote that they're waiting for the file descriptor to become readable or writeable.

然后，也许我希望这些条件变为真时，顶层循环能够恢复我的读写功能(以及顶层循环与它们之间的整个中间链).

And then, maybe I want that top level loop to be able to resume my read and write functions (and the whole intermediate chain between the top level loop and them) once those conditions become true.

我已经知道或多或少地使用 asyncio的方法.我写了这个

I already know how to use asyncio more or less. I wrote this little demo program that computes squares after a delay but launches lots of those tasks that each append to a list after a random interval. It's kind of clumsily written, but it works.

我想确切地知道该程序在做什么.为了做到这一点，我必须知道等待睡眠的时间如何通知顶级事件循环它想要睡眠(并再次调用)一会儿，以及调用之间所有中间堆栈帧的状态如何进入睡眠状态，顶层事件循环被冻结到位，然后在延迟结束后重新激活.

I want to know exactly what that program is doing under the hood. And in order to do that, I have to know how await on that sleep informs the top-level event loop that it wants to sleep (and be called again) for a bit and how the state of all the intermediate stack frames between the call to sleep and the top level event loop are frozen in place then reactivated when the delay is over.

推荐答案

因此，我更好地理解了如何使自己想做的事情变得更好.这是我的代码应阅读的方式:

So, I understand a lot better how to make what I was trying to do work. This is how my code should've read:

import types

@types.coroutine
def foo(x):
    yield x
    yield x + 1

async def intermediary(y):
    await foo(y)

def bar():
    c = intermediary(5)
    try:
        while True:
            result = c.send(None)
            print(f"Got {result} from the coroutine.")
    except StopIteration as e:
        print(f"StopIteration exception: {e!r}")

基本答案是，其端点可以是装饰有types.coroutine的普通生成器.有更多的方法可以完成这项工作，而我对代码的进一步修改向他们展示了它们:

The basic answer is that the endpoint of this can be a normal generator decorated with types.coroutine. There are more ways of making this work, and this further modification of my code demonstrates them:

import types
from collections.abc import Awaitable

@types.coroutine
def foo(x):
    sent = yield x
    print(f"foo was sent {sent!r}.")
    sent = yield x + 1
    print(f"foo was sent {sent!r}.")
    return 'generator'

class MyAwaitable(Awaitable):
    def __init__(self, x):
        super().__init__()
        self.x_ = x
    def __await__(self):
        def gen(x):
            for i in range(x-1, x+2):
                sent = yield i
                print(f"MyAwaitable was sent {sent!r}.")
            return 'class'
        return iter(gen(self.x_))

async def intermediary(t, y):
    awaited = await t(y)
    print(f"Got {awaited!r} as value from await.")

def runco(chain_end):
    c = intermediary(chain_end, 5)
    try:
        sendval = None
        while True:
            result = c.send(sendval)
            print(f"Got {result} from the coroutine.")
            sendval = sendval + 1 if sendval is not None else 0
    except StopIteration as e:
        print(f"StopIteration exception: {e!r}")

如您所见，任何定义返回迭代器的__await__方法的内容都可以使用await.真正发生的情况是，迭代await的对象被迭代直到停止为止，然后await返回.这样做的原因是，链末端的最后一件事可能会遇到某种阻塞情况.然后，它可以通过yield或从迭代器返回值(基本上与yield ing相同)来报告该情况(或要求设置回调或进行其他操作).然后，顶层循环可以继续执行其他任何可以运行的事情.

As you can see, anything that defines an __await__ method that returns an iterator can also be awaited upon. What really happens is that the thing being awaited upon is iterated over until it stops and then the await returns. The reason you do this is that the final thing at the end of the chain may encounter some kind of blocking condition. It can then report on that condition (or ask a callback to be set or something else) by yielding or returning a value from the iterator (basically the same thing as yielding). Then the top level loop can continue on to whatever other thing can be run.

整个await调用链的本质是，当您返回并从迭代器中请求下一个值时(回调到被阻止的函数中，告知它可能现在未被阻止).调用堆栈被重新激活.整个链的存在是在调用被阻止时保留调用堆栈状态的一种方式.基本上，是一个线程自动放弃控制，而不是由调度程序从中夺取控制权的线程.

The nature of the whole chain of await calls is that when you then go back and ask for the next value from the iterator (call back into the blocked function telling it that maybe it isn't blocked now) the entire call stack is reactivated. This whole chain exists as a way to preserve the state of the call stack while the call is blocked. Basically a thread that voluntarily gives up control rather than having control wrested from it by a scheduler.

当我问这个问题时，我脑海中关于asyncio在内部如何工作的愿景显然是一种叫做 curio的东西可以正常工作，并且基于端点例程yield来指示它们被阻止的内容以及正在运行的所有内容的顶级循环(在我的示例中为runco)，然后将其放入寻找某种通用条件池，以便一旦条件被更改阻止，它就可以恢复例程.在asyncio中，发生的事情要复杂得多，它使用带有__await__方法的对象(例如在我的示例中为MyAwaitable)和某种回调机制来使它们全部起作用.

The vision in my head of how asyncio worked internally when I asked this question is apparently how something called curio works and is based on the end point routines yielding some sort of indicator of what they're being blocked by and the top level loop that's running it all (runco in my example) then putting that in some sort of general pool of conditions to look for so it can resume the routine as soon as the condition it's blocked by changes. In asyncio, something much more complex happens, and it uses objects with the __await__ method (like MyAwaitable in my example) and some sort of callback mechanism to make it all work.

布雷特·坎农(Brett Cannon)写了一篇非常好的文章，谈论了生成器如何演变为协程.它会比我在StackOverflow答案中所能介绍的更为详尽.

Brett Cannon wrote a really good article that talks about how generators evolved into coroutines. It will go into far more detail than I can go into in a StackOverflow answer.

我发现的一个有趣的花絮是，当您这样做时:

One interesting tidbit I discovered is that when you do this:

def foo(x):
    yield 11

bar = types.coroutine(foo)

foo和bar都成为协程"，并且可以await打开.装饰者所做的一切只是在foo.__code__.co_flags中翻转一点.当然，这是一个实现细节，不应依赖于此.我认为这实际上是一个错误，我可能会这样报告.

Both foo and bar become 'coroutines' and can be awaited on. All the decorator does is flip a bit in foo.__code__.co_flags. This is, of course, an implementation detail and should not be relied upon. I think this is something of a bug actually, and I may report it as such.

这篇关于异步和等待关键字到底是如何工作的?等待链的末尾是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！