消息队列体系结构(客户端到Web服务器再到工作服务器再返回) [英] Message queue architecture (client to web server to worker and back)

问题描述

我在Heroku上运行了一个用Node JS编写的Web服务器.该服务器具有一个Web服务器进程和一个工作进程. Web服务器通过RabbitMQ队列成功将消息发送给工作程序；工作人员成功将处理后的数据返回到Web服务器.我使用随机生成的Uuid来跟踪消息，并确保将正确的消息与正确的原始消息配对.

I have a web server written in Node JS running on Heroku. The server has a web server process and a worker process. The web server successfully sends messages to the worker via a RabbitMQ queue; the worker successfully returns the processed data to the web server. I use a randomly generated Uuid to track the messages and ensure the right message is paired up with the right original message.

在一个单独的项目中，我让客户端(网站)成功与Web服务器通信.现在，我需要将两者放在一起.

In a separate project, I have the client (website) successfully communicating with the web server. Now, I need to put the two together.

如何做到这一点:

1. 客户端发送HTTP POST.
2. Web服务器接收请求并将请求传递到消息队列中.
3. 工作人员处理请求并返回到Web服务器.
4. Web服务器将正确的数据返回给客户端，以便客户端可以将响应与原始请求相关联.

第4步是我遇到的问题.我想我读过某个地方，客户端应该连续轮询(HTTP POST?)Web服务器，直到数据准备就绪为止.我认为我需要在第1步之后响应客户端，这样请求才不会超时.任何想法/建议表示赞赏！

Step 4 is where I'm stuck. I think I read somewhere that the client should continuously poll (HTTP POSTs?) the web server until it's data is ready. I think I need to respond to the client after step 1 so the request doesn't time out. Any ideas/advice appreciated!

框图:

推荐答案

您需要做的简短版本是双向消息传递.您的Web应用必须是消息产生者和消息使用者.后端服务也是如此.

The short version of what you need to do is two-way messaging. Your web app needs to be a message producer and a message consumer. The same goes for your back-end service.

当HTTP请求进入时，Web服务器通过RabbitMQ发送消息.后端在将来的某个时候将其提起.同时，Web服务器通过HTTP请求发送回响应，指出正在发生某些情况，稍后将通知用户.

When the HTTP request comes in, the web server sends a message through RabbitMQ. The back-end picks it up at some point in the future. Meanwhile, the web server sends a response back through the HTTP request saying something is happening and the user will be notified later.

如果您使用的是Express，它将看起来像这样:

If you're using express, it would look something like this:

var router = express.Router();
router.post("/", postJob);

function postJob(req, res, next){
  req.session.inProgress = true;

  var msg = {
    job: "do some work"
  };

  jobSender.sendJobRequest(msg, function(err){
    if (err) { return next(err); }
    res.render("some-response");
  });
}

此代码有很多假设，例如jobSender是某种封装对象，具有通过RabbitMQ发送消息的方法.我确定您可以根据您已经说过的内容填写发送邮件的详细信息.

This code makes a lot of assumptions, like jobSender being some kind of encapsulated object with a method to send a message across RabbitMQ. I'm sure you can fill in the details of sending a message, based on what you've said already.

重要的是，HTTP请求处理程序通过RabbitMQ发送消息，然后将HTTP响应发送回Web浏览器.

The important thing, is that the HTTP request handler sends the message across RabbitMQ and then sends an HTTP response back to the web browser.

此时，浏览器可以执行所需的任何操作.

At this point, the browser can do whatever it needs to do.

在后端，当另一项服务完成工作时，它将需要执行以下两项操作之一:

On the back-end, when the other service has completed it's work, it will need to do one of two things:

1)在某处更新共享数据库，以便您的Web服务器知道已完成的工作(并可以读取状态)

1) update a shared database somewhere, so that your web server knows what work has been done (and can read a status)

或

2)通过Rabbitmq将消息发送回Web服务器

2) send a message back to the web server, via rabbitmq

选项#1出于各种原因可能并不总是一个好的选择.并且从您的问题中，您还是想要选项#2.

option #1 might not always be a good option, for various reasons. and from your question, you want option #2 anyways.

您将需要第二个队列-Web服务器正在侦听的队列.当Web服务器从该队列接收到消息时，它将使用接收状态更新其自己的数据库.

you'll need a second queue - one that the web server is listening to. when the web server receives a message from this queue, it will update it's own database with the status that is receives.

此状态可能是完成"，进行中"或错误"，或者您认为合适的其他状态.

this status may be "complete" or "in progress" or "error" or something else that you see fit.

例如，如果您有一条工作状态"消息，则可能有一个称为"JobStatusReceiver"的抽象来接收状态消息.

For example, if you have a "job status" message, you may have an abstraction called "JobStatusReceiver" to receive the status message.

像这样的简单模块可以从您的工作状态队列接收消息，并使用状态更新本地数据库

A simple module like this could receive messages from your job status queue, and update a local database with the status

var JobStatusReceiver = require("./jobStatusReceiver");
var someDataObject = require("someDataObject");

var jobStatus = {

  listen: function(){
    var receiver = new JobStatusReceiver();
    receiver.receive(function(statusMessage){

      someDataObject.status = statusMessage.status;
      someDataObject.data = statusMessage.data;


      someDataObject.save();
    });
  }

};

module.exports = jobStatus;

请注意，这可能在Web服务器中发生，但这不是HTTP请求的一部分.该消息是通过带有JobStatusReceiver的RabbitMQ传入的，而不是HTTP请求的一部分.

note that this may be happening in the web server, but it's not part of an HTTP request. the message came in through RabbitMQ with teh JobStatusReceiver, and not part of an HTTP request.

someDataObject对象很可能是yoru数据库中的对象，因此可以将其保存回数据库中.

the someDataObject object is most likely going to be an object from yoru database, so it can be saved back to the database.

最后，您需要通过数据将完成的操作通知用户的部分可以通过多种方式发生.

Lastly, the part where you need to notify the user of the completed action, with the data, can happen in a number of ways.

通常来说，每隔几秒钟对Web服务器上的HTTP API进行AJAX调用很容易，以寻找有效的响应.

Generally speaking, it's fairly easy to make an AJAX call to an HTTP API on your web server every few seconds, to look for a valid response.

在浏览器端，这可能很简单:

On the browser side, this could be as simple as:

var timer = setInterval(function(){

  $.ajax({
    url: "/api/check-status",
    success: function(data){
      if (data.complete){
        clearInterval(timer);
        doSomeOtherWork(data);
      }
    })
  });

});

然后在Express应用程序中再次处理"/api/check-status"，您将使用相同的"someDataObject"模型来检查状态:

and again in the Express app, handling the "/api/check-status", you would use the same "someDataObject" model to check the status:

var someDataObject = require("someDataObject");

var router = new express.Router();
router.get("/", checkStatus);

function checkStatus(req, res, next){

  someDataObject.load(function(err, someData){
    if (err) { return next(err); }

    res.json({
      complete: (someData.status === "complete"),
      data: someData.data
    });

  });

}

这有望使您走上正确的道路.当然，我遗漏了很多细节，但希望您能够填写缺失的部分.

This should hopefully get you down the right path. There are a lot of details I've left out, of course, but hopefully you'll be able to fill in the missing parts.

...

PS:除了我的 RabbitMQ 4 Devs 培训课程.这是一个完整的程序包，可用于RabbitMQ和Node.js并运行

P.S.: I cover all of this, except for the browser checking for status updates on a timer, in my RabbitMQ 4 Devs training course. It's a complete package to get up and running with RabbitMQ and Node.js

这篇关于消息队列体系结构(客户端到Web服务器再到工作服务器再返回)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！