Heroku上的节点JS消息队列 [英] Node JS message queue on Heroku

问题描述

我需要将运行在Heroku上的Node JS服务器移动到消息队列体系结构中。目前，服务器接收到HTTP请求，进行一些处理并作出响应。问题是处理需要一些时间，特别是当有很多请求时。这种冗长的处理时间会导致服务器超时，超载和崩溃！我的阅读告诉我需要一个后台工作人员来处理。

我对消息队列和后台工作者没有任何经验，我正在寻找一个非常简单的例子来开始。任何人都可以提出一个简单易懂的模块或示例来开始吗？

我发现了一些例子，但它们很复杂，我迷路了！我想要一个我可以建立的准系统例子。

解决方案

让我们来看看如何用RabbitMQ来做到这一点。
首先，您需要在开发环境中使用RabbitMQ服务器。
如果你还没有安装它（检查sudo service rabbitmq-server status），你可以安装（在Ubuntu或类似的）如下：

 sudo su -cecho'deb http://www.rabbitmq.com/debian/ testing main'>> /etc/apt/sources.list
 wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc 
 sudo apt-key add rabbitmq-signing-key-public.asc 
 sudo apt-get update 
 sudo apt-get install rabbitmq-server 
 rm rabbitmq-signing-key-public.asc 
  

然后，让服务器运行：

  sudo服务rabbitmq-server start 
  

您还需要为您的Heroku部署配置RabbitMQ服务。在这个例子中，我们使用CloudAMPQ。您可以将其免费计划添加到您的Heroku应用程序中：

  heroku addons：create cloudamqp：lemur 
  

这会在您的Heroku应用程序中创建一个新的CLOUDAMQP_URL环境变量。

接下来，您将需要为您的node.js应用程序提供合适的RabbitMQ客户端。
这里有一些，但是在这个例子中，我们使用ampqlib：

  npm install ampqlib  - -save 
  

这应该在package.json依赖关系中添加如下代码行：

 amqplib：^ 0.4.1，
  

接下来是为您的Heroku应用程序添加一个背景workerdyno。
我假设你目前在Procfile中只有一个Web dyno。
因此，您需要添加另一行来实例化一个worker，例如：

  worker：node myworker.js 
  

最后，您需要编写代码使您的Web dyno能够通过工作dyno RabbitMQ。

为了这个例子，我将假设你的Web dyno将把消息发布到一个RabbitMQ消息队列中，并且你的工人dyno将会是消费这些消息。

因此，让我们开始编写发布到消息队列的代码。这段代码需要在你的Web dyno的某个地方运行：

  //定义ampq_url指向Heroku上的CLOUDAMPQ_URL或本地RabbitMQ服务器在开发环境中
 var ampq_url = process.env.CLOUDAMQP_URL || AMQP：// localhost 的; 
 var ampq_open = require（'amqplib'）; 
 var publisherChnl; 
 
函数createPublisherChannel（）{
 
 //创建一个AMPQ连接
 ampq_open.connect（ampq_url）
 .then（function（conn） {
 //你需要在你的连接上创建至少一个AMPQ频道
 var ok = conn.createChannel（）; 
 ok = ok.t​​hen（function（ch）{
 publisherChnl = ch; 
 //现在为要发送给工作人员的实际消息创建一个队列dyno 
 publisherChnl.assertQueue（'my-worker-q'）; 
} ）
}）
} 
 
函数publishMsg（）{
 //向工作人员发送消息
 publisherChnl.sendToQueue（'my-worker-q '，新缓冲区（'来自Web dyno的Hello World'））; 
} 
  

您需要在初始化Web dyno期间调用createPublisherChannel（）。然后，当你想发送消息到队列时调用publishMsg（）。

最后，让我们编写在worker dyno中使用上述消息的代码。因此，例如，在myworker.js中添加如下内容：

  //就像在Web dyno中一样... 
 var amqp_url = process.env.CLOUDAMQP_URL || AMQP：// localhost 的; 
 var open_ampq = require（'amqplib'）。connect（amqp_url）; 
 var consumerChnl; 
 
 //创建一个AMPQ频道，用于在'my-worker-q'上消费消息
函数createConsumerChannel（）{
 open_ampq 
 .then（function（conn） （
 conn.createChannel（）
 .then（function（ch）{
 ch.assertQueue（'my-worker-q'）; 
 consumerChnl = ch; 
}）; 
}）; 
 
 
函数startConsuming（）{
 consumerChnl.consume（'my-worker-q'，function（msg）{
 if（msg！== null ）{
 console.log（msg.content.toString（））; 
 //告诉RabbitMQ服务器我们已经消费了消息
 consumerChnl.ack（msg）; 
} 
}）
} 
 
 createConsumerChnl（）。then（startConsuming）; 
  

最后，用heroku local进行测试。你应该看到你现在有两个进程在你的应用中运行，Web和worker。无论何时在您的Web dyno中调用publishMsg（），您都应该看到wroker dyno将消息内容吐出到您的控制台。要查看您的RabbitMQ队列中发生了什么，您可以使用：

  sudo rabbitmqctl list_queues 
  

I need to move my Node JS server running on Heroku to a message queue architecture. Currently, the server receives a HTTP request, does some processing, and responds. The problem is that the processing takes some time, especially when there are lots of requests. This lengthy processing time causes the server to timeout, overload, and crash! My reading tells me a need a background worker to do the processing.

I have zero experience with message queues and background workers and I'm looking for a very simple example to get started. Can anyone suggest a simple, understandable module or example to get started?

I found some examples but they are complex and I'm getting lost! I want a barebones example I can build from.

解决方案

Let's see how to do this with RabbitMQ. First, you will need a RabbitMQ server to work with in your development environment. If you don't already have it (check "sudo service rabbitmq-server status") you can install (on ubuntu or similar) as follows:

sudo su -c "echo 'deb http://www.rabbitmq.com/debian/ testing main' >> /etc/apt/sources.list"
wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
sudo apt-key add rabbitmq-signing-key-public.asc
sudo apt-get update
sudo apt-get install rabbitmq-server
rm  rabbitmq-signing-key-public.asc

Then, get the server running with:

sudo service rabbitmq-server start

You also need to configure a RabbitMQ service for your Heroku deployment. Let's use CloudAMPQ for this example. You can add its Free Plan to your Heroku app with:

heroku addons:create cloudamqp:lemur 

That will create a new CLOUDAMQP_URL environment variable in your Heroku app.

Next, you're going to need a suitable RabbitMQ client for your node.js app. There are a few of them out there, but for this example, let's use ampqlib:

npm install ampqlib --save

That should add something like the following line in your package.json dependencies:

"amqplib": "^0.4.1",

Next thing is to add a background "worker" dyno to your Heroku app. I assume that currently you only have a single Web dyno in your Procfile. So, you need to add another line for instantiating a worker, such as:

worker: node myworker.js

Finally, you need to write the code that will enable your Web dyno to interact with your worker dyno via RabbitMQ.

For the sake of this example, I will assume that your Web dyno will be "publishing" messages to a RabbitMQ message queue, and your worker dyno will be "consuming" these messages.

So, let's start with writing code for publishing to a message queue. This code needs to run somewhere in your Web dyno:

// Define ampq_url to point to CLOUDAMPQ_URL on Heroku, or local RabbitMQ server in dev environment
var ampq_url = process.env.CLOUDAMQP_URL || "amqp://localhost";
var ampq_open = require('amqplib');
var publisherChnl;

function createPublisherChannel() {

    // Create an AMPQ "connection"
    ampq_open.connect(ampq_url)
        .then(function(conn) {
            // You need to create at least one AMPQ "channel" on your connection   
            var ok = conn.createChannel();
            ok = ok.then(function(ch){
                publisherChnl = ch;
                // Now create a queue for the actual messages to be sent to the worker dyno 
                publisherChnl.assertQueue('my-worker-q');
            })
        })
    }

function publishMsg() {
     // Send the worker a message
     publisherChnl.sendToQueue('my-worker-q', new Buffer('Hello world from Web dyno'));
}

You will need to call createPublisherChannel() during the initialisation of your Web dyno. Then, call publishMsg() whenever you want to send a message to the queue.

Finally, let's write the code for consuming the above message in the worker dyno. So, for example, add something like the following in myworker.js:

// Just like in Web dyno...
var amqp_url = process.env.CLOUDAMQP_URL || "amqp://localhost";
var open_ampq = require('amqplib').connect(amqp_url);
var consumerChnl;    

// Creates an AMPQ channel for consuming messages on 'my-worker-q'
function createConsumerChannel() {     
    open_ampq
        .then(function(conn) {
            conn.createChannel()
                .then(function(ch) {
                    ch.assertQueue('my-worker-q');
                    consumerChnl = ch;
            });
        });
}  

function startConsuming() {
    consumerChnl.consume('my-worker-q', function(msg){
        if (msg !== null) {
            console.log(msg.content.toString());
            // Tell RabbitMQ server we have consumed the message
            consumerChnl.ack(msg);
        }
    })
} 

createConsumerChnl().then(startConsuming); 

Finally, test with "heroku local". You should see that you now have 2 processes running in your app, "Web" and "worker". Whenever you call publishMsg() in your Web dyno, you should hopefully see the wroker dyno spit out the message contents to your console. To see what's happening in your RabbitMQ queues, you can use:

sudo rabbitmqctl list_queues

这篇关于Heroku上的节点JS消息队列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！