Python asyncio / aiohttp：有关BaseProtocol.connection_lost（）的要求是什么？ [英] Python asyncio/aiohttp: What are the requirements regarding BaseProtocol.connection_lost()?

问题描述

针对<的python 文档 code> connection_lost 指出：

connection_made（）和connection_lost（）每次成功调用一次

另外还有以下状态机：

开始-> connection_made（）[-> data_received（）*] [-> eof_received（）吗？]-> connection_lost（）->结束

此外， BaseTransport.close（）的文档指出：

刷新缓冲的数据后，将以无为参数调用协议的connection_lost（）方法。

和文档，用于 WriteTransport。 abort（）状态：

该协议的connection_lost（）方法最终将以None为参数被调用。

在我看来，这表明以下职责：

1. 传输必须，如果它已调用 connection_made（），则以后还要调用 connection_lost（）上的协议（无论是否由于调用 close（），调用 abort（ ）或基础连接有问题）。


2. 协议一定不能假定在调用以下命令时I / O已完成 close（）或 abort（）返回。它必须等待对 connection_lost（）的调用。特别是，在 close（）或 abort（）返回之后，可能还有一些与运输计划有关的工作

请牢记这一点，请考虑以下使用SSL的普通aiohttp客户端程序：

  import aiohttp 
导入asyncio 
 
 async def main（）：
 conn = aiohttp.TCPConnector（verify_ssl = False）
与aiohttp.ClientSession（connector = conn）作为会话异步：
与session.post（'https：// whatevs /'）作为异步：
 resp.raise_for_status（）
 
 loop = asyncio.get_event_loop（）
 try：
 loop.run_until_complete（main（））
最后：
 loop.close（）
  

在我的（windows）机器上运行此命令似乎可以正常工作
正确地。但是，如果我将断点或打印语句放入
connection_made（）和 connection_lost（）方法中aiohttp的
ResponseHandler 类（协议实现），我看到
connection_made（）是

使用的传输方式为 _SSLProtocolTransport ，但没有调用 connection_lost（）。 c $ c>，在asyncio的
sslproto.py 文件中定义。它的 close（）方法 被调用，并在关闭过程中启动
。由于SSL的性质，该关闭过程是
必然是异步的，并且期望的是，一旦
关闭完成， SSLProtocol
_SSLProtocolTransport 将通过其 _finalize（）方法关闭
its 基础传输。然后，这将导致对
connection_lost 的调用使堆栈膨胀。但是，这些
异步操作实际上没有发生。 aiohttp似乎只是调用
close（）并立即丢弃 _SSLProtocolTransport （
方法在这里它甚至都不是协程，并且传输
永远不会按照其关闭顺序进行，也永远不会调用
connection_lost（）。

所以我的问题是：这是aiohttp和/或aysncio的SSL
协议/传输中的错误，还是我误解了文档，因为
认为传输和协议的职责？

为什么要问这个问题

这个问题的原因是我编写了自己的SSL传输
，以允许我将PyOpenSSL与asyncio一起使用，而不是
标准库 ssl 模块。在我的实现中，在我的 close（）方法对
的调用返回之后，在
事件循环中仍然有排队的回调（排定为 call_soon（））。为了使
能够正确执行异步关闭序列，这是必需的，并且
我希望协议为我的传输提供机会完成
进程并调用 connection_lost（）。

当我将运输工具与aiohttp一起使用时， __ aexit __ 方法在上面的代码中创建的
ClientSession 调用其自己的 close（）
方法（而不是协程），这导致我的运输被关闭，
而没有等待 connection_lost（）。然后关闭事件循环
，并在传输仍处于活动状态且
执行I / O时完成该模块，从而导致各种错误。

<我试图找出这是我的错还是aiohttp
中的错误（也许还有asyncio的SSL传输）。如果是我的错，我需要
知道我应该如何执行此异步关机。我
原则上可以通过运行事件
循环直到调用loop.close（）之前为空，在顶层处理它，但是我看不到任何
的方式可以做到这一点（有 Task.all_tasks（），但这不适用于通过 call_soon 安排的
事情。即使我可以以某种方式做到这一点，它在某些情况下也显得丑陋，并且在我为asyncio或其他人见过的任何
文档中，肯定没有将其描述为关闭后的
标准要求。 aiohttp。

解决方案

我建议您在aiohttp错误跟踪器中创建一个问题并将其复制到该问题中。
IMHO Stack Overflow不是讨论此类问题的最佳场所。

The python documentation for connection_lost states:

connection_made() and connection_lost() are called exactly once per successful connection.

Further down there's also the following state machine:

start -> connection_made() [-> data_received() *] [-> eof_received() ?] -> connection_lost() -> end

Also, the documentation for BaseTransport.close() states:

After all buffered data is flushed, the protocol’s connection_lost() method will be called with None as its argument.

and the documentation for WriteTransport.abort() states:

The protocol’s connection_lost() method will eventually be called with None as its argument.

This seems to me to indicate the following responsibilities:

1. The transport must, if it has called connection_made(), later also call connection_lost() on the protocol (regardless of whether the connection is lost because of a call to close(), a call to abort() or an issue with the underlying connection).
2. The protocol must not assume that I/O has finished when a call to close() or abort() returns. It must wait for the call to connection_lost(). In particular, after close() or abort() returns, there may be work relating to the transport still scheduled on the event loop.

With that in mind, consider the following trivial aiohttp client program, using SSL:

import aiohttp
import asyncio

async def main():
    conn = aiohttp.TCPConnector(verify_ssl=False)
    async with aiohttp.ClientSession(connector=conn) as session:
        async with session.post('https://whatevs/') as resp:
            resp.raise_for_status()

loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(main())
finally:
    loop.close()

Running this on my (windows) machine appears to work correctly. However, if I put breakpoints or print statements into the connection_made() and connection_lost() methods of aiohttp's ResponseHandler class (a protocol implementation), I see that connection_made() is called but connection_lost() is not.

The transport used is _SSLProtocolTransport, defined in asyncio's sslproto.py file. Its close() method is called, and it sets off a shutdown process. Due to the nature of SSL this shutdown process is necessarily asynchronous, and the expectation appears to be that once the shutdown is complete the SSLProtocol underlying the _SSLProtocolTransport would, from its _finalize() method, close its underlying transport. This would then cause a call to connection_lost to bubble up the stack. However, none of this asynchronous stuff actually happens. aiohttp appears to just call close() and immediately discard the _SSLProtocolTransport (the method where it does this is not even a coroutine), and the transport never progresses with its shutdown sequence and never calls connection_lost().

So my question is: is this a bug in aiohttp and/or aysncio's SSL protocol/transport, or am I misinterpreting the documentation as regards the responsitilities of the transport and protocol?

Why I'm Asking This

The reason for this question is that I have written an SSL transport of my own, to allow me to use PyOpenSSL with asyncio, instead of the standard library ssl module. In my implementation, after the call to my close() method returns, there are still callbacks queued on the event loop (scheduled with call_soon()). This is necessary in order for the asynchronous shutdown sequence to be performed correctly, and I expect the protocol to give my transport a chance to complete the process and call connection_lost().

When I use my transport with aiohttp, the __aexit__ method of the ClientSession created in the code above calls its own close() method (not a coroutine), which causes my transport to be closed, without waiting for connection_lost(). The event loop is then closed and the module finalised while the transport is still alive and performing I/O, resulting in a variety of errors.

I'm trying to figure out whether this is my fault or a bug in aiohttp (and perhaps also asyncio's SSL transport). If it's my fault, I need to know how I'm supposed to perform this asynchronous shutdown. I could in principle handle it at the top level by running the event loop until it's empty before calling loop.close(), but I don't see any way to do that (there's Task.all_tasks() but that doesn't work for things scheduled with call_soon). Even if I can do that somehow, it would seem exceptionally ugly and is certainly not described as a standard requirement for shutting down after such work in any documentation I've seen for asyncio or aiohttp.

解决方案

I suggest you to create an issue in aiohttp bug tracker and copy your question into it. IMHO Stack Overflow is not the best place for discussing questions like this.

这篇关于Python asyncio / aiohttp：有关BaseProtocol.connection_lost（）的要求是什么？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！