更新客户端芹菜任务已完成 [英] Updating client that a celery task has finished

问题描述

我有以下要求.我正在创建一个Web应用程序(无前端)，该应用程序将接收帖子并获取请求.其功能如下.客户将在URL上发布帖子，从而触发一项大型且耗时的任务.客户还可以发出Get请求并检索任务的结果.

I have the following requirement. I am creating a web app (no frontend) that will receive post and get requests. It's functionality will be the following. Client will make a post on a URL triggering a large and time expensive task. A Client will also have the ability to make a Get request and retrieve the results of the taks.

我当时正在考虑使用Celery启动任务.任务是非常大的算术计算.我的问题是如何通知客户他触发的任务已经完成并且他可以检索结果.

I was thinking of using Celery to start the task. The task is a very large arithmetic computation. My problem is on how to notify the client that the task he triggered has finished and he can retrieve the result.

所以我是这样想的:

client ->posts to web app -> web app starts task to celery -> request closed

web app notifies -> client triggered task  # what is the proper way

client -> get request to web app -> web app returns result

我知道没有一种正确的方法，所以我会更具体一些.目前，该任务将由另一个烧瓶应用程序自动触发.有没有一种方法可以自动通知Flask应用程序任务已完成，并获取结果并将其存储在自己的数据库中?第二个问题是，即使客户端是Flask应用程序，Android应用程序，IOS应用程序等，哪一种最通用的解决方案仍然有效?

I know there isn't one proper way, so I' ll be a bit more specific. For the time being the task will be triggered by another flask application, automatically. Is there a way the flask application to be automatically notified that the task is finished and get the results and store them in its own database? And a second question what would be the most generic solution that would work even if the client is a flask app, an android app an IOS app etc.

我之所以问的原因是因为我在辩论一个api是否将结果发布回客户端，或者客户端应该负责检查并获取结果.

The reason I also ask is because I had a debate on whether an api post the results back to the client or the client should be responsible to check and get the results.

推荐答案

过去，我不得不做出类似的决定，即在完成一项长期运行的芹菜任务后，需要更新客户端.设计系统以解决问题的方式有很多:

I've had to make a similar decision in the past, where a client needed to be updated upon completion of a long-running celery task. There are many different ways to design the system to solve the problem:

1. 具有轮询的状态端点-这是最简单的解决方案.在Flask中公开一个端点，该端点返回作业的状态，并简单地从客户端进行轮询.但是，有几件事要注意，即客户数量和轮询频率.如果客户端太多或轮询太频繁，则可能会遇到性能问题.如果您有几个客户并且不需要即时状态更新，那么这是理想的选择.
2. WebSockets -在客户端和Flask服务器之间打开一个WebSocket，并在任务完成后使用它推送更新.这种方法将在多个客户端上更好地扩展，并允许近乎即时的通知.
3. 回调URL -您提到客户端是Flask应用.您可以在客户端服务器上公开回调路由，并在启动任务时将其提供给celery.任务完成后，Celery可以发布到回调URL，这将通知客户端.

1. Status Endpoint with Polling - This is the simplest solution. Expose an endpoint in flask which returns the status of a job and simply poll from the client. There are several things to be mindful of however, namely the number of clients and frequency of polling. If you have too many clients or are polling too frequently, you could run into performance issues. If you have a few clients and don't need instant status updates, this is ideal.
2. WebSockets - Open up a WebSocket between the client and flask server, and use this to push updates when your task is complete. This approach will scale better with multiple clients and will allow for near instant notifications.
3. Callback URL - You mentioned that the client was a Flask app. You could expose a callback route on the client server and provide this to celery when starting the task. Upon task completion, Celery can POST to the callback URL and this will notify the client.

对于最通用的解决方案，它可以是状态端点，也可以是WebSockets-Android&iOS具有内置的HTTP功能，可以使用库处理WebSocket.另一方面，回调URL解决方案特定于Flask.

As for the most generic solution, that would be either the status endpoint or WebSockets - Android & iOS have HTTP functionality built in and can handle WebSockets using libraries. The callback URL solution on the other hand is specific to Flask.

这篇关于更新客户端芹菜任务已完成的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！