使用集群将 Socket.IO 扩展到多个 Node.js 进程 [英] Scaling Socket.IO to multiple Node.js processes using cluster

问题描述

用这个把我的头发扯掉……有没有人设法将 Socket.IO 扩展到由 Node.js 产生的多个工作"进程.js的集群模块?

Tearing my hair out with this one... has anyone managed to scale Socket.IO to multiple "worker" processes spawned by Node.js's cluster module?

假设我在四个工作进程(伪)上有以下内容:

Lets say I have the following on four worker processes (pseudo):

// on the server
var express = require('express');
var server = express();
var socket = require('socket.io');
var io = socket.listen(server);

// socket.io
io.set('store', new socket.RedisStore);

// set-up connections...
io.sockets.on('connection', function(socket) {

  socket.on('join', function(rooms) {
    rooms.forEach(function(room) {
      socket.join(room);
    });
  });

  socket.on('leave', function(rooms) {
    rooms.forEach(function(room) {
      socket.leave(room);
    });
  });

});

// Emit a message every second
function send() {
  io.sockets.in('room').emit('data', 'howdy');
}

setInterval(send, 1000);

在浏览器上...

// on the client
socket = io.connect();
socket.emit('join', ['room']);

socket.on('data', function(data){
  console.log(data);
});

问题:由于四个独立的工作进程发送消息，我每秒收到四条消息.

The problem: Every second, I'm receiving four messages, due to four separate worker processes sending the messages.

如何确保消息只发送一次?

How do I ensure the message is only sent once?

推荐答案

在 Socket.IO 1.0+ 中，与其设置一个具有多个 Redis 客户端的存储，一个更简单的 Redis 适配器模块现在可以使用.

In Socket.IO 1.0+, rather than setting a store with multiple Redis clients, a simpler Redis adapter module can now be used.

var io = require('socket.io')(3000);
var redis = require('socket.io-redis');
io.adapter(redis({ host: 'localhost', port: 6379 }));

下面显示的示例看起来更像这样:

The example shown below would look more like this:

var cluster = require('cluster');
var os = require('os');

if (cluster.isMaster) {
  // we create a HTTP server, but we do not use listen
  // that way, we have a socket.io server that doesn't accept connections
  var server = require('http').createServer();
  var io = require('socket.io').listen(server);
  var redis = require('socket.io-redis');

  io.adapter(redis({ host: 'localhost', port: 6379 }));

  setInterval(function() {
    // all workers will receive this in Redis, and emit
    io.emit('data', 'payload');
  }, 1000);

  for (var i = 0; i < os.cpus().length; i++) {
    cluster.fork();
  }

  cluster.on('exit', function(worker, code, signal) {
    console.log('worker ' + worker.process.pid + ' died');
  }); 
}

if (cluster.isWorker) {
  var express = require('express');
  var app = express();

  var http = require('http');
  var server = http.createServer(app);
  var io = require('socket.io').listen(server);
  var redis = require('socket.io-redis');

  io.adapter(redis({ host: 'localhost', port: 6379 }));
  io.on('connection', function(socket) {
    socket.emit('data', 'connected to worker: ' + cluster.worker.id);
  });

  app.listen(80);
}

如果你有一个主节点需要发布到其他Socket.IO进程，但本身不接受socket连接，使用socket.io-emitter 而不是 socket.io-redis.

If you have a master node that needs to publish to other Socket.IO processes, but doesn't accept socket connections itself, use socket.io-emitter instead of socket.io-redis.

如果您在扩展时遇到问题，请使用 DEBUG=* 运行您的 Node 应用程序.Socket.IO 现在实现了 debug，它也会打印出 Redis 适配器调试消息.示例输出:

If you are having trouble scaling, run your Node applications with DEBUG=*. Socket.IO now implements debug which will also print out Redis adapter debug messages. Example output:

socket.io:server initializing namespace / +0ms
socket.io:server creating engine.io instance with opts {"path":"/socket.io"} +2ms
socket.io:server attaching client serving req handler +2ms
socket.io-parser encoding packet {"type":2,"data":["event","payload"],"nsp":"/"} +0ms
socket.io-parser encoded {"type":2,"data":["event","payload"],"nsp":"/"} as 2["event","payload"] +1ms
socket.io-redis ignore same uid +0ms

如果您的主进程和子进程都显示相同的解析器消息，那么您的应用程序正在正确扩展.

If both your master and child processes both display the same parser messages, then your application is properly scaling.

如果您是从单个工作人员发出的，那么您的设置应该没有问题.您正在做的是从所有四个工作人员发出，并且由于 Redis 发布/订阅，消息不会重复，而是按照您要求应用程序的方式写入四次.这是Redis所做的简单图表:

There shouldn't be a problem with your setup if you are emitting from a single worker. What you're doing is emitting from all four workers, and due to Redis publish/subscribe, the messages aren't duplicated, but written four times, as you asked the application to do. Here's a simple diagram of what Redis does:

Client  <--  Worker 1 emit -->  Redis
Client  <--  Worker 2  <----------|
Client  <--  Worker 3  <----------|
Client  <--  Worker 4  <----------|

如你所见，当你从一个worker发出时，它会将这个emit发布到Redis，并且会从其他订阅Redis数据库的worker镜像.这也意味着你可以使用多个连接到同一个实例的套接字服务器，并且一个服务器上的发射将在所有连接的服务器上被触发.

As you can see, when you emit from a worker, it will publish the emit to Redis, and it will be mirrored from other workers, which have subscribed to the Redis database. This also means you can use multiple socket servers connected the the same instance, and an emit on one server will be fired on all connected servers.

使用集群，当客户端连接时，它将连接到您的四个工作人员之一，而不是全部四个.这也意味着您从该工作人员发出的任何内容都只会向客户端显示一次.所以是的，应用程序正在扩展，但是你这样做的方式，你从所有四个工作人员发出，Redis 数据库使它好像你在一个工作人员上调用它四次.如果客户端实际上连接到您的所有四个套接字实例，他们将每秒接收 16 条消息，而不是 4 条.

With cluster, when a client connects, it will connect to one of your four workers, not all four. That also means anything you emit from that worker will only be shown once to the client. So yes, the application is scaling, but the way you're doing it, you're emitting from all four workers, and the Redis database is making it as if you were calling it four times on a single worker. If a client actually connected to all four of your socket instances, they'd be receiving sixteen messages a second, not four.

套接字处理的类型取决于您将拥有的应用程序类型.如果您要单独处理客户端，那么您应该没有问题，因为连接事件只会为每个客户端的一个工作人员触发.如果您需要全局心跳"，那么您可以在主进程中有一个套接字处理程序.由于工人在主进程死亡时死亡，因此您应该抵消主进程的连接负载，并让子进程处理连接.举个例子:

The type of socket handling depends on the type of application you're going to have. If you're going to handle clients individually, then you should have no problem, because the connection event will only fire for one worker per one client. If you need a global "heartbeat", then you could have a socket handler in your master process. Since workers die when the master process dies, you should offset the connection load off of the master process, and let the children handle connections. Here's an example:

var cluster = require('cluster');
var os = require('os');

if (cluster.isMaster) {
  // we create a HTTP server, but we do not use listen
  // that way, we have a socket.io server that doesn't accept connections
  var server = require('http').createServer();
  var io = require('socket.io').listen(server);

  var RedisStore = require('socket.io/lib/stores/redis');
  var redis = require('socket.io/node_modules/redis');

  io.set('store', new RedisStore({
    redisPub: redis.createClient(),
    redisSub: redis.createClient(),
    redisClient: redis.createClient()
  }));

  setInterval(function() {
    // all workers will receive this in Redis, and emit
    io.sockets.emit('data', 'payload');
  }, 1000);

  for (var i = 0; i < os.cpus().length; i++) {
    cluster.fork();
  }

  cluster.on('exit', function(worker, code, signal) {
    console.log('worker ' + worker.process.pid + ' died');
  }); 
}

if (cluster.isWorker) {
  var express = require('express');
  var app = express();

  var http = require('http');
  var server = http.createServer(app);
  var io = require('socket.io').listen(server);

  var RedisStore = require('socket.io/lib/stores/redis');
  var redis = require('socket.io/node_modules/redis');

  io.set('store', new RedisStore({
    redisPub: redis.createClient(),
    redisSub: redis.createClient(),
    redisClient: redis.createClient()
  }));

  io.sockets.on('connection', function(socket) {
    socket.emit('data', 'connected to worker: ' + cluster.worker.id);
  });

  app.listen(80);
}

在示例中，有五个 Socket.IO 实例，一个是主实例，四个是子实例.主服务器从不调用 listen()，因此该进程没有连接开销.但是，如果您在主进程上调用一个发射，它将被发布到 Redis，四个工作进程将在它们的客户端上执行发射.这会将连接负载抵消给工作人员，如果工作人员死亡，则主应用程序中的主要应用程序逻辑将保持不变.

In the example, there are five Socket.IO instances, one being the master, and four being the children. The master server never calls listen() so there is no connection overhead on that process. However, if you call an emit on the master process, it will be published to Redis, and the four worker processes will perform the emit on their clients. This offsets connection load to workers, and if a worker were to die, your main application logic would be untouched in the master.

请注意，对于 Redis，即使在命名空间或房间中的所有发出都将由其他工作进程处理，就好像您从该进程触发了发出一样.换句话说，如果您有两个 Socket.IO 实例和一个 Redis 实例，则在第一个工作线程上的套接字上调用 emit() 会将数据发送到其客户端，而第二个工作线程将执行相同操作就像你从那个工人那里调用了发射一样.

Note that with Redis, all emits, even in a namespace or room will be processed by other worker processes as if you triggered the emit from that process. In other words, if you have two Socket.IO instances with one Redis instance, calling emit() on a socket in the first worker will send the data to its clients, while worker two will do the same as if you called the emit from that worker.

这篇关于使用集群将 Socket.IO 扩展到多个 Node.js 进程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！