长时间运行的REST API与队列 [英] Long running REST API with queues

问题描述

我们正在实施一个REST API，这将揭开序幕多长时间运行的后台任务。我一直在阅读了REST Web服务菜谱和推荐是返回HTTP 202 /与内容位置头指向任务正在处理接受。 （例如： http://www.example.org/orders/tasks/1234 ），并让客户端轮询这个URI有关长期运行的任务更新。

We are implementing a REST API, which will kick off multiple long running backend tasks. I have been reading the RESTful Web Services Cookbook and the recommendation is to return HTTP 202 / Accepted with a Content-Location header pointing to the task being processed. (e.g. http://www.example.org/orders/tasks/1234), and have the client poll this URI for an update on the long running task.

这个想法是有REST API立即发布消息到一个队列，与背景工作者角色从队列中拿起消息和纺纱多个后台任务，也使用队列。我用这种方法看到的问题是如何将一个唯一的ID分配给该任务，随后让客户机请求的任务的状态通过发出一个GET到内容 - 位置的URI。

The idea is to have the REST API immediately post a message to a queue, with a background worker role picking up the message from the queue and spinning up multiple backend tasks, also using queues. The problem I see with this approach is how to assign a unique ID to the task and subsequently let the client request a status of the task by issuing a GET to the Content-Location URI.

如果对REST API立即职位，以队列，那么它可以生成一个GUID，并附上为被添加到队列中的消息的属性，但在获取请求的状态变得很尴尬。

If the REST API immediately posts to a queue, then it could generate a GUID and attach that as an attribute on the message being added to the queue, but fetching the status of the request becomes awkward.

另一种选择是有REST API立即条目添加到数据库中（假设订单和新订单ID），一个初始状态，并随后将消息队列揭开序幕后地面任务，然后将随后更新数据库记录。该API将在Content-Location头的URI返回新订单ID，客户端检查任务的状态时使用。

Another option would be to have the REST API immediately add an entry to the database (let's say an order, with a new order id), with an initial status and then subsequently put a message on the queue to kick off the back ground tasks, which would then subsequently update that database record. The API would return this new order ID in the URI of the Content-Location header, for the client to use when checking the status of the task.

不知何故首先将数据库条目，然后添加消息到队列似乎向后，但只加入请求到队列使得它难以跟踪进展。

Somehow adding the database entry first, then adding the message to the queue seems backwards, but only adding the request to the queue makes it hard to track progress.

什么是建议的方法？

非常感谢您的见解。

推荐答案

我假设你的系统看起来像下面这样。你有一个REST服务，该服务从客户端接收请求。它转换请求到其中的业务逻辑可以理解的命令。你把这些命令放到一个队列。你有一个或多个人员可处理并从队列中删除这些命令，并将结果发送到REST服务，可以响应客户端。

I assume your system looks like the following. You have a REST service, which receives requests from the client. It converts the requests into commands which the business logic can understand. You put these commands into a queue. You have a single or multiple workers which can process and remove these commands from the queue and send the results to the REST service, which can respond to the client.

您的问题，您的长时间运行任务的客户端连接超时，所以你不能发送响应。所以你能做的就是送你把命令后到队列中，并添加一个轮询链接202所接受，因此客户端将能查询的变化。你的任务具有多个子任务，以便有进步，不只是悬而未决，完整的状态变化。

Your problem that by your long running tasks the client connection timeouts, so you cannot send a response. So what you can do is sending a 202 accepted after you put the commands into the queue and add a polling link, so the client will be able to poll for the changes. Your tasks have multiple subtasks so there is progress, not just pending and complete status changes.

1. 如果你想坚持投票，你应该创建一个新的REST资源，其中包含的实际状态和长时间运行的任务的进度。这意味着，你必须存储在数据库这个信息，所以REST服务将能够像 GET /任务/ 23461 /状态请求作出回应。这意味着你的工人有当它完成子任务或整个任务来更新数据库。


2. 如果您的REST服务作为守护程序运行，那么你可以通过进度通知，所以在数据库中存储任务的状态不会是工人的责任。这种REST服务的可存储在存储器中的信息，以及


3. 如果您决定使用WebSockets来通知客户端，那么你可以创建一个通知服务。休息后，你必须有一个任务ID响应。之后，你的WebSocket连接上发送回这个任务ID，所以通知服务就会知道哪些WebSocket连接订阅了某项任务的事件。之后，你将不再需要REST服务，可以通过WebSocket连接发送的进度，只要客户端不关闭连接。


4. 您可以将这些解决方案通过以下方式。你让你的REST服务创建任务的资源，所以你就可以通过使用轮询链接访问的进展情况。之后，您发回与您发回通过的WebSockets连接202的标识符。所以，你可以使用通知服务来通知客户端。通过进度你的工人将通知REST服务，这将创建一个像链接GET /任务/ 23461 /状态，并通过通知服务发送的链接到客户端。在此之后，客户端可以使用链接来更新其状态。

1. If you want to stick with polling, you should create a new REST resource, which contains the actual state and the progress of the long running task. This means that you have to store this info in a database, so the REST service will be able to respond to requests like GET /tasks/23461/status. This means that your worker has to update the database when it is completed a subtask or the whole task.
2. If your REST service is running as a daemon, then you can notify it by progress, so storing the task status in the database won't be the responsibility of the worker. This kind of REST service can store the info in the memory as well.
3. If you decide to use websockets to notify the client, then you can create a notification service. By REST you have to respond with a task id. After that you send back this task id on the websocket connection, so the notification service will know which websocket connection subscribed to the events of a certain task. After that you won't need the REST service, you can send the progress through the websocket connection as long as the client does not close the connection.
4. You can combine these solutions the following way. You let your REST service to create a task resource, so you'll be able to access the progress by using a polling link. After that you send back an identifier with 202 which you send back through the websockets connection. So you can use a notification service to notify the client. By progress your worker will notify the REST service, which will create a link like GET /tasks/23461/status and send that link to the client through the notification service. After that the client can use the link to update its status.

我觉得最后一个是最好的解决办法，如果你的REST服务作为一个守护进程运行。这是因为你可以将通知负责一个专门的通知服务，可以使用WebSockets，投票，SSE，任何你想要的。它可以折叠而不杀死REST服务，因此REST服务将保持稳定，快速。如果用202也发回手动更新链接，然后在客户端可以做手动更新（假设人类控制客户端），这样你才会有像优雅降级如果通知服务不可用。您不必维护通知服务，因为它不会知道任务什么，它只是将数据发送到客户端。你的工人将不必知道如何发送通知，以及如何创建超链接的任何东西。这将是更容易维护客户code过，因为这将是一个几乎纯粹的REST客户端。唯一的额外功能将是通知链接订阅，这并不经常更改。

I think the last one is the best solution if your REST service runs as a daemon. It is because you can move the notification responsibility to a dedicated notification service, which can use websockets, polling, SSE, whatever you want. It can collapse without killing the REST service, so the REST service will stay stable and fast. If you send back a manual update link too with the 202, then the client can do manual update (assuming a human controlled client), so you will have something like graceful degradation if the notification service is not available. You don't have to maintain the notification service because it won't know anything about the tasks, it will just send data to the clients. Your worker won't have to know anything about how to send notifications and how to create hyperlinks. It will be easier to maintain the client code too, since it will be almost a pure REST client. The only extra feature will be the subscription for the notification links, which does not change frequently.

这篇关于长时间运行的REST API与队列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！