使用来自 Celery 任务的 SocketIO 进行服务器推送 [英] Server Push with SocketIO from Celery Task

问题描述

我有一个 flask 应用程序，其中有许多长时间运行的异步任务(~小时).与客户端沟通这些任务的状态很重要.

I have a flask application within which I have many long running asynchronous tasks (~hours). It's important that the state of these tasks is communicated with the client.

我使用 celery 来管理后台任务队列，我目前正在尝试通过 socketIO 从每个后台线程向客户端广播更新.这可能吗?是否有更适合的策略来实现我的愿望?

I use celery to manage the background task queue, and I'm currently trying to broadcast updates to the client from each background thread via socketIO. Is this possible? Is there a better suited strategy to achieving what I would like?

推荐答案

你没有说，但我假设你打算使用 Flask-SocketIO 来处理服务器端 SocketIO 而不是官方的 Node.js 服务器，对吗?

You did not say, but I assume you plan on using Flask-SocketIO to handle the server-side SocketIO and not the official Node.js server, correct?

你想做的事情可以做，但是对于当前版本的 Flask-SocketIO，问题是承载 Flask 和 Flask-SocketIO 服务器的进程拥有与客户端的套接字连接，所以它是唯一的可以与它们通信的进程.此时，Flask-SocketIO 不提供任何帮助从其他进程(例如 Celery worker)向客户端发送数据，这部分您必须自己实现.特别是对于 Celery，您可以让长时间运行的任务公开服务器进程可以获取并发送给客户端的进度信息.

What you want to do can be done, but with the current version of Flask-SocketIO, the problem is that the process that hosts the Flask and Flask-SocketIO server owns the socket connections with the clients, so it is the only process that can communicate with them. At this time, Flask-SocketIO does not offer any help in sending data to clients from other processes such as Celery workers, this part you have to implement yourself. Specifically for Celery, you can have your long running tasks expose progress information that the server process can pick up and send to the clients.

我目前正在改进 Flask-SocketIO，这将使任何进程能够使用 Redis 发布/订阅后端向连接的客户端发送消息，以便与 Flask-SocketIO 服务器进行通信.完成这项工作后，您将能够从 Celery 进程透明地将数据写入任何客户端.

I am currently working on improvements to Flask-SocketIO that will enable any process to send messages to connected clients using a Redis pub/sub backend for communication to the Flask-SocketIO server. Once this work is completed you will be able to write data to any client transparently from your Celery process.

您还询问是否还有其他选择.您还应该考虑客户端可以轮询服务器的状态.如果更新不需要非常频繁，那么这是一个更容易实施的选项.客户端向服务器询问给定任务的状态，服务器反过来询问 Celery 任务.我在我的 Flask+Celery 博客文章中展示了这种方法.

You also ask if there is another alternative. You should also consider that the client can poll the server for status. If the updates do not need to be very frequent, then this is an option that is going to be much easier to implement. The client asks the server for status for a given task, and the server in turn asks the Celery task. I showed this approach in my Flask+Celery blog article.

这篇关于使用来自 Celery 任务的 SocketIO 进行服务器推送的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！