Python Webserver:如何异步处理请求 [英] Python Webserver: How to serve requests asynchronously

问题描述

我需要创建一个将执行以下操作的python中间件:

I need to create a python middleware that will do the following:

a)接受来自多个客户端的http get/post请求.

a) Accept http get/post requests from multiple clients.

b)修改这些请求并将其分发到后端远程应用程序(通过套接字通信).我对该远程应用程序没有任何控制权.

b) Modify and Dispatch these requests to a backend remote application (via socket communication). I do not have any control over this remote application.

c)从后端应用程序接收处理后的结果，并将这些结果返回给发出请求的客户端.

c) Receive processed results from backend application and return these results back to the requesting clients.

现在，客户端期望同步请求/响应方案.但是后端应用程序不会同步返回结果.也就是说，某些请求比其他请求花费更长的时间.因此，

Now the clients are expecting a synchronous request/response scenario. But the backend application is not returning the results synchronously. That is, some requests take much longer to process than others. Hence,

客户端1:发送http请求C1->获取响应R1

Client 1 : send http request C1 --> get response R1

客户端2:发送http请求C2->获取响应R2

Client 2 : send http request C2 --> get response R2

客户端3:发送http请求C3->获取响应R3

Client 3 : send http request C3 --> get response R3

Python中间件以某种顺序接收它们:C2，C3，C1.按此顺序将它们调度到后端(作为非HTTP消息).后端以混合顺序R1，R3，R2的结果进行响应.Python中间件应将这些响应打包回http响应对象，然后将响应发送回相关的客户端.

Python middleware receives them in some order: C2, C3, C1. Dispatches them in this order to backend (as non-http messages). Backend responds with results in mixed order R1, R3, R2. Python middleware should package these responses back into http response objects and send the response back to the relevant client.

是否有任何示例代码可以对这种行为进行编程.似乎有20种不同的python网络框架，我对于哪种方案最适合这种情况感到困惑(我希望尽可能轻量级的东西……我会认为Django太重了……我尝试了bottle，但我不确定如何针对这种情况进行编程).

Is there any sample code to program this sort of behavior. There seem to be something like 20 different web frameworks for python and I'm confused as to which one would be best for this scenario (would prefer something as lightweight as possible ... I would consider Django too heavy ... I tried bottle, but I am not sure how to go about programming that for this scenario).

=================================================

================================================

更新(基于下面的讨论):请求具有请求ID.响应具有响应ID(应与它们对应的请求ID相匹配).中间件和远程后端应用程序之间只有一个套接字连接.虽然我们可以维护{request_id:ip_address}字典，但问题是如何为正确的客户端构造HTTP响应对象.我认为，线程可以解决每个线程都维护自己的响应对象的问题.

Update (based on discussions below): Requests have a request id. Responses have a response id (which should match the request id that they correspond to). There is only one socket connection between the middleware and the remote backend application. While we can maintain a {request_id : ip_address} dictionary, the issue is how to construct a HTTP response object to the correct client. I assume, threading might solve this problem where each thread maintains its own response object.

推荐答案

螺丝框架.这正是 asyncore 的任务.该模块允许基于事件的网络编程:给定一组套接字，当其中任何一个数据就绪时，它将回调给定的处理程序.这样，线程就不必只是呆滞地等待一个套接字上的数据到达并将其痛苦地传递给另一个线程.您必须自己实现http处理，但是可以找到一些示例.另外，您可以使用 uwsgi 的异步功能，该功能可将您的应用程序与现有的Web服务器，但默认情况下不会与asyncore集成---尽管使其工作并不困难.取决于特定需求.

Screw frameworks. This exactly the kind of task for asyncore. This module allows event-based network programming: given a set of sockets, it calls back given handlers when data is ready on any of them. That way, threads are not necessary just to dumbly wait for data on one socket to arrive and painfully pass it to another thread. You would have to implement the http handling yourself, but examples can be found on that. Alternatively, you could use the async feature of uwsgi, which would allow your application to be integrated with an existing webserver, but that does not integrate with asyncore by default --- though it wouldn't be hard to make it work. Depends on specific needs.

这篇关于Python Webserver:如何异步处理请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！