asyncio.gather的顺序版本 [英] Sequential version of asyncio.gather

问题描述

我试图创建一种类似于asyncio.gather的方法，但是该方法将按顺序而不是异步地执行任务列表:

I tried to create a method similar to asyncio.gather, but which will execute the list of tasks sequentially and not asynchronously:

async def in_sequence(*tasks):
    """Executes tasks in sequence"""
    for task in tasks:
        await task

接下来应该像这样使用这种方法:

Next this method was supposed to be used like this:

async def some_work(work_name):
    """Do some work"""
    print(f"Start {work_name}")
    await asyncio.sleep(1)
    if raise_exception:
        raise RuntimeError(f"{work_name} raise an exception")
    print(f"Finish {work_name}")

async def main():
    try:
        await asyncio.gather(
            some_work("work1"),         # work1, work2, in_sequence and work5 executed in concurrently
            some_work("work2"),
            in_sequence(
                some_work("work3"),     # work3 and work4 executed in sequence
                some_work("work4")
            ),
            some_work("work5"),


    except RuntimeError as error:
        print(error)                    # raise an exception at any point to terminate

一切正常，直到我尝试在some_work中抛出异常:

And everything worked fine until I tried to throw an exception in some_work:

async def main():
    try:
        await asyncio.gather(
            some_work("work1"),
            some_work("work2"),
            in_sequence(
                some_work("work3", raise_exception=True),       # raise an exception here
                some_work("work4")
            ),
            some_work("work5"),


    except RuntimeError as error:
        print(error)

此后，我立即收到以下错误消息:

Immediately after that, I received the following error message:

RuntimeWarning: coroutine 'some_work' was never awaited

我阅读了文档并继续进行实验:

I read the documentation and continued to experiment:

async def in_sequence(*tasks):
    """Executes tasks in sequence"""
    _tasks = []
    for task in tasks:
        _tasks.append(asyncio.create_task(task))

    for _task in _tasks:
        await _task

此版本按预期工作！

在这方面，我还有其他问题:

In this regard, I have next questions:

1. 为什么第二个版本有效，而第一个版本无效?
2. asyncio是否已经具有执行任务列表的工具 顺序地?
3. 我选择了正确的实施方法还是有更好的方法 选项?

1. Why does the second version work and the first not?
2. Does asyncio already have the tools to execute the list of tasks sequentially?
3. Have I chosen the right implementation method or are there better options?

推荐答案

此版本按预期工作！

And this version worked as expected!

第二个版本的问题是它实际上没有顺序运行协程，而是并行运行它们.这是因为asyncio.create_task()将协程安排为与当前协程并行运行.因此，当您循环等待任务时，实际上是在等待第一个任务的同时允许所有任务运行.尽管有出现，但整个循环的运行时间仅与最长的任务一样长. (有关详细信息，请参见此处.)

The problem with the second version is that it doesn't actually run the coroutines sequentially, it runs them in parallel. This is because asyncio.create_task() schedules the coroutine to run in parallel with the current coroutines. So when you await tasks in a loop, you are actually allowing all the tasks to run while awaiting the first one. Despite appearances, the whole loop will run for only as long as the longest task. (See here for more details.)

第一个版本显示的警告旨在防止您意外创建从未等待的协程，例如只写asyncio.sleep(1)而不是await asyncio.sleep(1).就异步而言，main正在实例化协程对象并将它们传递给in_sequence，该对象忘记了"以等待其中的一些对象.

The warning displayed by your first version is intended to prevent you from accidentally creating a coroutine that you never await, e.g. writing just asyncio.sleep(1) instead of await asyncio.sleep(1). As far as asyncio is concerned, main is instantiating coroutine objects and passing them to in_sequence which "forgets" to await some of them.

抑制警告消息的一种方法是允许协程旋转，但立即将其取消.例如:

One way to suppress the warning message is to allow the coroutine to spin, but cancel it immediately. For example:

async def in_sequence(*coros):
    remaining = iter(coros)
    for coro in remaining:
        try:
            await coro
        except Exception:
            for c in remaining:
                asyncio.create_task(c).cancel()
            raise

请注意，以下划线开头的变量名称会标记为未使用的变量，因此您不应命名变量，以免您确实使用了它们.

Note that a variable name that begins with an underscore marks an unused variable, so you shouldn't name variables so if you actually do use them.

这篇关于asyncio.gather的顺序版本的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！