Heroku 上的 Node JS 消息队列 [英] Node JS message queue on Heroku

问题描述

我需要将在 Heroku 上运行的 Node JS 服务器移动到消息队列架构.目前，服务器接收 HTTP 请求，进行一些处理，然后进行响应.问题是处理需要一些时间，特别是当有很多请求时.这种冗长的处理时间会导致服务器超时、过载和崩溃！我的阅读告诉我需要一个后台工作人员来进行处理.

I need to move my Node JS server running on Heroku to a message queue architecture. Currently, the server receives a HTTP request, does some processing, and responds. The problem is that the processing takes some time, especially when there are lots of requests. This lengthy processing time causes the server to timeout, overload, and crash! My reading tells me a need a background worker to do the processing.

我对消息队列和后台工作人员的经验为零，我正在寻找一个非常简单的示例来开始使用.任何人都可以建议一个简单易懂的模块或示例来入门吗?

I have zero experience with message queues and background workers and I'm looking for a very simple example to get started. Can anyone suggest a simple, understandable module or example to get started?

我发现了一些示例但它们很复杂，我迷路了！我想要一个可以构建的准系统示例.

I found some examples but they are complex and I'm getting lost! I want a barebones example I can build from.

推荐答案

让我们看看如何用 RabbitMQ 做到这一点.首先，您需要在开发环境中使用 RabbitMQ 服务器.如果您还没有它(检查sudo service rabbitmq-server status")，您可以安装(在 ubuntu 或类似设备上)，如下所示:

Let's see how to do this with RabbitMQ. First, you will need a RabbitMQ server to work with in your development environment. If you don't already have it (check "sudo service rabbitmq-server status") you can install (on ubuntu or similar) as follows:

sudo su -c "echo 'deb http://www.rabbitmq.com/debian/ testing main' >> /etc/apt/sources.list"
wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
sudo apt-key add rabbitmq-signing-key-public.asc
sudo apt-get update
sudo apt-get install rabbitmq-server
rm  rabbitmq-signing-key-public.asc

然后，让服务器运行:

sudo service rabbitmq-server start

您还需要为您的 Heroku 部署配置一个 RabbitMQ 服务.让我们在此示例中使用 CloudAMPQ.您可以通过以下方式将其免费计划添加到您的 Heroku 应用程序中:

You also need to configure a RabbitMQ service for your Heroku deployment. Let's use CloudAMPQ for this example. You can add its Free Plan to your Heroku app with:

heroku addons:create cloudamqp:lemur 

这将在您的 Heroku 应用中创建一个新的 CLOUDAMQP_URL 环境变量.

That will create a new CLOUDAMQP_URL environment variable in your Heroku app.

接下来，您将需要一个适合您的 node.js 应用程序的 RabbitMQ 客户端.其中有一些，但对于这个例子，让我们使用 ampqlib:

Next, you're going to need a suitable RabbitMQ client for your node.js app. There are a few of them out there, but for this example, let's use ampqlib:

npm install ampqlib --save

这应该在你的 package.json 依赖项中添加类似以下行的内容:

That should add something like the following line in your package.json dependencies:

"amqplib": "^0.4.1",

接下来是向您的 Heroku 应用程序添加一个后台worker"dyno.我假设目前您的 Procfile 中只有一个 Web dyno.所以，你需要添加另一行来实例化一个worker，比如:

Next thing is to add a background "worker" dyno to your Heroku app. I assume that currently you only have a single Web dyno in your Procfile. So, you need to add another line for instantiating a worker, such as:

worker: node myworker.js

最后，您需要编写代码，使您的 Web dyno 能够通过 RabbitMQ 与您的 worker dyno 进行交互.

Finally, you need to write the code that will enable your Web dyno to interact with your worker dyno via RabbitMQ.

就本示例而言，我将假设您的 Web dyno 将发布"消息到 RabbitMQ 消息队列，而您的 worker dyno 将消耗"这些消息.

For the sake of this example, I will assume that your Web dyno will be "publishing" messages to a RabbitMQ message queue, and your worker dyno will be "consuming" these messages.

那么，让我们从编写用于发布到消息队列的代码开始.此代码需要在您的 Web dyno 中的某个位置运行:

So, let's start with writing code for publishing to a message queue. This code needs to run somewhere in your Web dyno:

// Define ampq_url to point to CLOUDAMPQ_URL on Heroku, or local RabbitMQ server in dev environment
var ampq_url = process.env.CLOUDAMQP_URL || "amqp://localhost";
var ampq_open = require('amqplib');
var publisherChnl;

function createPublisherChannel() {

    // Create an AMPQ "connection"
    ampq_open.connect(ampq_url)
        .then(function(conn) {
            // You need to create at least one AMPQ "channel" on your connection   
            var ok = conn.createChannel();
            ok = ok.then(function(ch){
                publisherChnl = ch;
                // Now create a queue for the actual messages to be sent to the worker dyno 
                publisherChnl.assertQueue('my-worker-q');
            })
        })
    }

function publishMsg() {
     // Send the worker a message
     publisherChnl.sendToQueue('my-worker-q', new Buffer('Hello world from Web dyno'));
}

您需要在 Web dyno 初始化期间调用 createPublisherChannel().然后，只要您想向队列发送消息，就调用 publishMsg().

You will need to call createPublisherChannel() during the initialisation of your Web dyno. Then, call publishMsg() whenever you want to send a message to the queue.

最后，我们在worker dyno中编写消费上述消息的代码.因此，例如，在 myworker.js 中添加如下内容:

Finally, let's write the code for consuming the above message in the worker dyno. So, for example, add something like the following in myworker.js:

// Just like in Web dyno...
var amqp_url = process.env.CLOUDAMQP_URL || "amqp://localhost";
var open_ampq = require('amqplib').connect(amqp_url);
var consumerChnl;    

// Creates an AMPQ channel for consuming messages on 'my-worker-q'
function createConsumerChannel() {     
    open_ampq
        .then(function(conn) {
            conn.createChannel()
                .then(function(ch) {
                    ch.assertQueue('my-worker-q');
                    consumerChnl = ch;
            });
        });
}  

function startConsuming() {
    consumerChnl.consume('my-worker-q', function(msg){
        if (msg !== null) {
            console.log(msg.content.toString());
            // Tell RabbitMQ server we have consumed the message
            consumerChnl.ack(msg);
        }
    })
} 

createConsumerChnl().then(startConsuming); 

最后，使用heroku local"进行测试.您应该看到您的应用程序中现在有 2 个进程在运行，Web"和worker".每当您在 Web dyno 中调用 publishMsg() 时，您应该希望看到 wroker dyno 将消息内容吐出到您的控制台.要查看 RabbitMQ 队列中发生的情况，您可以使用:

Finally, test with "heroku local". You should see that you now have 2 processes running in your app, "Web" and "worker". Whenever you call publishMsg() in your Web dyno, you should hopefully see the wroker dyno spit out the message contents to your console. To see what's happening in your RabbitMQ queues, you can use:

sudo rabbitmqctl list_queues

这篇关于Heroku 上的 Node JS 消息队列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！