使用集群将Socket.IO扩展到多个Node.js进程 [英] Scaling Socket.IO to multiple Node.js processes using cluster

问题描述

用这个撕掉我的头发......有人设法将 Socket.IO 扩展到Node生成的多个工人进程。 js的群组模块？

Tearing my hair out with this one... has anyone managed to scale Socket.IO to multiple "worker" processes spawned by Node.js's cluster module?

让我说我有以下内容四个工作进程（伪）：

Lets say I have the following on four worker processes (pseudo):

// on the server
var express = require('express');
var server = express();
var socket = require('socket.io');
var io = socket.listen(server);

// socket.io
io.set('store', new socket.RedisStore);

// set-up connections...
io.sockets.on('connection', function(socket) {

  socket.on('join', function(rooms) {
    rooms.forEach(function(room) {
      socket.join(room);
    });
  });

  socket.on('leave', function(rooms) {
    rooms.forEach(function(room) {
      socket.leave(room);
    });
  });

});

// Emit a message every second
function send() {
  io.sockets.in('room').emit('data', 'howdy');
}

setInterval(send, 1000);

在浏览器上......

And on the browser...

// on the client
socket = io.connect();
socket.emit('join', ['room']);

socket.on('data', function(data){
  console.log(data);
});

问题：每一秒，我收到四消息，由于四个单独的工作进程发送消息。

The problem: Every second, I'm receiving four messages, due to four separate worker processes sending the messages.

如何确保消息仅发送一次？

How do I ensure the message is only sent once?

推荐答案

编辑：在Socket.IO 1.0+中，不是设置多个Redis客户端的商店，现在可以使用更简单的Redis适配器模块使用。

In Socket.IO 1.0+, rather than setting a store with multiple Redis clients, a simpler Redis adapter module can now be used.

var io = require('socket.io')(3000);
var redis = require('socket.io-redis');
io.adapter(redis({ host: 'localhost', port: 6379 }));

下面显示的示例更像是这样：

The example shown below would look more like this:

var cluster = require('cluster');
var os = require('os');

if (cluster.isMaster) {
  // we create a HTTP server, but we do not use listen
  // that way, we have a socket.io server that doesn't accept connections
  var server = require('http').createServer();
  var io = require('socket.io').listen(server);
  var redis = require('socket.io-redis');

  io.adapter(redis({ host: 'localhost', port: 6379 }));

  setInterval(function() {
    // all workers will receive this in Redis, and emit
    io.emit('data', 'payload');
  }, 1000);

  for (var i = 0; i < os.cpus().length; i++) {
    cluster.fork();
  }

  cluster.on('exit', function(worker, code, signal) {
    console.log('worker ' + worker.process.pid + ' died');
  }); 
}

if (cluster.isWorker) {
  var express = require('express');
  var app = express();

  var http = require('http');
  var server = http.createServer(app);
  var io = require('socket.io').listen(server);
  var redis = require('socket.io-redis');

  io.adapter(redis({ host: 'localhost', port: 6379 }));
  io.on('connection', function(socket) {
    socket.emit('data', 'connected to worker: ' + cluster.worker.id);
  });

  app.listen(80);
}

如果您有一个需要发布到其他Socket.IO进程的主节点，但不接受套接字连接本身，使用 socket.io-emitter 而不是< a href =https://github.com/automattic/socket.io-redis =noreferrer> socket.io-redis 。

If you have a master node that needs to publish to other Socket.IO processes, but doesn't accept socket connections itself, use socket.io-emitter instead of socket.io-redis.

如果您在扩展时遇到问题，请使用 DEBUG = * 运行您的节点应用程序。 Socket.IO现在实现调试，它还将打印出Redis适配器调试消息。示例输出：

If you are having trouble scaling, run your Node applications with DEBUG=*. Socket.IO now implements debug which will also print out Redis adapter debug messages. Example output:

socket.io:server initializing namespace / +0ms
socket.io:server creating engine.io instance with opts {"path":"/socket.io"} +2ms
socket.io:server attaching client serving req handler +2ms
socket.io-parser encoding packet {"type":2,"data":["event","payload"],"nsp":"/"} +0ms
socket.io-parser encoded {"type":2,"data":["event","payload"],"nsp":"/"} as 2["event","payload"] +1ms
socket.io-redis ignore same uid +0ms

如果主进程和子进程都显示相同的解析器消息，那么您的应用程序正在正确扩展。

If both your master and child processes both display the same parser messages, then your application is properly scaling.

如果您从单个工作人员身上散发，您的设置应该没有问题。您正在做的是从所有四个工作人员中发出的，并且由于Redis发布/订阅，消息不会重复，而是写入四次，就像您要求应用程序执行的那样。以下是Redis所做的简单图表：

There shouldn't be a problem with your setup if you are emitting from a single worker. What you're doing is emitting from all four workers, and due to Redis publish/subscribe, the messages aren't duplicated, but written four times, as you asked the application to do. Here's a simple diagram of what Redis does:

Client  <--  Worker 1 emit -->  Redis
Client  <--  Worker 2  <----------|
Client  <--  Worker 3  <----------|
Client  <--  Worker 4  <----------|

正如您所看到的，当您从工作人员发出时，它会将发布发布到Redis，并且它将与其他已订阅Redis数据库的工作人员进行镜像。这也意味着您可以使用连接相同实例的多个套接字服务器，并且将在所有连接的服务器上触发一个服务器上的emit。

As you can see, when you emit from a worker, it will publish the emit to Redis, and it will be mirrored from other workers, which have subscribed to the Redis database. This also means you can use multiple socket servers connected the the same instance, and an emit on one server will be fired on all connected servers.

使用群集时，当客户端时连接，它将连接到你的四个工人之一，而不是所有四个工人。这也意味着您从该工作人员发出的任何内容只会向客户显示一次。所以，是的，应用程序正在扩展，但是你正在这样做，你从所有四个工作人员中散发出来，而Redis数据库就像你在一个工人上调用它四次一样。如果客户端实际连接到所有四个套接字实例，则它们每秒接收十六条消息，而不是四条消息。

With cluster, when a client connects, it will connect to one of your four workers, not all four. That also means anything you emit from that worker will only be shown once to the client. So yes, the application is scaling, but the way you're doing it, you're emitting from all four workers, and the Redis database is making it as if you were calling it four times on a single worker. If a client actually connected to all four of your socket instances, they'd be receiving sixteen messages a second, not four.

套接字处理的类型取决于类型您将拥有的应用程序。如果您要单独处理客户端，那么您应该没有问题，因为连接事件将仅针对每个客户端的一个工作程序触发。如果您需要全局心跳，那么您可以在主进程中拥有套接字处理程序。由于工作程序在主进程终止时死亡，因此应该从主进程中抵消连接负载，并让子进程处理连接。这是一个例子：

The type of socket handling depends on the type of application you're going to have. If you're going to handle clients individually, then you should have no problem, because the connection event will only fire for one worker per one client. If you need a global "heartbeat", then you could have a socket handler in your master process. Since workers die when the master process dies, you should offset the connection load off of the master process, and let the children handle connections. Here's an example:

var cluster = require('cluster');
var os = require('os');

if (cluster.isMaster) {
  // we create a HTTP server, but we do not use listen
  // that way, we have a socket.io server that doesn't accept connections
  var server = require('http').createServer();
  var io = require('socket.io').listen(server);

  var RedisStore = require('socket.io/lib/stores/redis');
  var redis = require('socket.io/node_modules/redis');

  io.set('store', new RedisStore({
    redisPub: redis.createClient(),
    redisSub: redis.createClient(),
    redisClient: redis.createClient()
  }));

  setInterval(function() {
    // all workers will receive this in Redis, and emit
    io.sockets.emit('data', 'payload');
  }, 1000);

  for (var i = 0; i < os.cpus().length; i++) {
    cluster.fork();
  }

  cluster.on('exit', function(worker, code, signal) {
    console.log('worker ' + worker.process.pid + ' died');
  }); 
}

if (cluster.isWorker) {
  var express = require('express');
  var app = express();

  var http = require('http');
  var server = http.createServer(app);
  var io = require('socket.io').listen(server);

  var RedisStore = require('socket.io/lib/stores/redis');
  var redis = require('socket.io/node_modules/redis');

  io.set('store', new RedisStore({
    redisPub: redis.createClient(),
    redisSub: redis.createClient(),
    redisClient: redis.createClient()
  }));

  io.sockets.on('connection', function(socket) {
    socket.emit('data', 'connected to worker: ' + cluster.worker.id);
  });

  app.listen(80);
}

在这个例子中，有五个Socket.IO实例，一个是主实例，四个是孩子。主服务器从不调用 listen（），因此该进程没有连接开销。但是，如果在主进程上调用emit，它将发布到Redis，并且四个工作进程将在其客户端上执行emit。这会将连接负载抵消给工人，如果工作人员死亡，主应用程序逻辑将在主服务器中不受影响。

In the example, there are five Socket.IO instances, one being the master, and four being the children. The master server never calls listen() so there is no connection overhead on that process. However, if you call an emit on the master process, it will be published to Redis, and the four worker processes will perform the emit on their clients. This offsets connection load to workers, and if a worker were to die, your main application logic would be untouched in the master.

请注意，使用Redis时，所有都会发出，甚至在命名空间或空间中，将由其他工作进程处理，就像您从该进程触发了emit一样。换句话说，如果你有两个带有一个Redis实例的Socket.IO实例，在第一个worker的套接字上调用 emit（）会将数据发送给它的客户端，同时worker 2将会像调用该worker的emit一样。

Note that with Redis, all emits, even in a namespace or room will be processed by other worker processes as if you triggered the emit from that process. In other words, if you have two Socket.IO instances with one Redis instance, calling emit() on a socket in the first worker will send the data to its clients, while worker two will do the same as if you called the emit from that worker.

这篇关于使用集群将Socket.IO扩展到多个Node.js进程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！