我在我的集​​群 node.js/socket.io/redis pub/sub 应用程序中收到重复的消息 [英] I'm receiving duplicate messages in my clustered node.js/socket.io/redis pub/sub application

问题描述

我正在使用 Node.js、Socket.io 和 Redisstore、来自 Socket.io 的集群和 Redis.

I'm using Node.js, Socket.io with Redisstore, Cluster from the Socket.io guys, and Redis.

我有一个发布/订阅应用程序，它仅在一个 Node.js 节点上运行良好.但是，由于 Node.js 不是为多核机器编写的，因此当它承受重负载时，只能最大限度地使用服务器的一个核心.

I've have a pub/sub application that works well on just one Node.js node. But, when it comes under heavy load is maxes out just one core of the server since Node.js isn't written for multi-core machines.

如下所示，我现在使用的是 Learnboost 的 Cluster 模块，他们是 Socket.io 的开发者.

As you can see below, I'm now using the Cluster module from Learnboost, the same people who make Socket.io.

但是，当我启动 4 个工作进程时，每个进来并订阅的浏览器客户端都会获得 Redis 中发布的每条消息的 4 个副本.如果有 3 个工作进程，则有 3 个副本.

But, when I fire up 4 worker processes, each browser client that comes in and subscribes gets 4 copies of each message that is published in Redis. If there are are three worker processes, there are three copies.

我猜我需要以某种方式将 redis 发布/订阅功能移动到 cluster.js 文件中.

I'm guessing I need to move the redis pub/sub functionality to the cluster.js file somehow.

Cluster.js

var cluster = require('./node_modules/cluster');

cluster('./app')
  .set('workers', 4)
  .use(cluster.logger('logs'))
  .use(cluster.stats())
  .use(cluster.pidfiles('pids'))
  .use(cluster.cli())
  .use(cluster.repl(8888))
  .listen(8000);

App.js

redis = require('redis'),
sys = require('sys');

var rc = redis.createClient();

var path = require('path')
  , connect = require('connect')
  , app = connect.createServer(connect.static(path.join(__dirname, '../')));

// require the new redis store
var sio = require('socket.io')
  , RedisStore = sio.RedisStore
  , io = sio.listen(app);

io.set('store', new RedisStore);io.sockets.on('connection', function(socket) {
    sys.log('ShowControl -- Socket connected: ' + socket.id);

    socket.on('channel', function(ch) {
        socket.join(ch)
        sys.log('ShowControl -- ' + socket.id + ' joined channel: ' + ch);
    });

    socket.on('disconnect', function() {
        console.log('ShowControll -- Socket disconnected: ' + socket.id);
    });
});

rc.psubscribe('showcontrol_*');

rc.on('pmessage', function(pat, ch, msg) {
    io.sockets.in(ch).emit('show_event', msg);
    sys.log('ShowControl -- Publish sent to channel: ' + ch);
});

// cluster compatiblity
if (!module.parent) {
  app.listen(process.argv[2] || 8081);
  console.log('Listening on ', app.address());
} else {
  module.exports = app;
}

client.html

<script src="http://localhost:8000/socket.io/socket.io.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.0/jquery.min.js"></script>
<script>
    var socket = io.connect('localhost:8000');
    socket.emit('channel', 'showcontrol_106');
    socket.on('show_event', function (msg) {
        console.log(msg);
        $("body").append('<br/>' + msg);
    });
</script>

推荐答案

事实证明这不是 Node.js/Socket.io 的问题，我只是以完全错误的方式去做.

Turns out this isn't a problem with Node.js/Socket.io, I was just going about it the completely wrong way.

我不仅从 Node/Socket 堆栈之外发布到 Redis 服务器，我还直接订阅了 Redis 频道.在发布/订阅情况的两端，我都绕过了后端带有 Redis 存储的 Socket.io 集群"的优点.

Not only was I publishing into the Redis server from outside the Node/Socket stack, I was still directly subscribed to the Redis channel. On both ends of the pub/sub situation I was bypassing the "Socket.io cluster with Redis Store on the back end" goodness.

因此，我创建了一个小应用程序(使用 Node.js/Socket.io/Express)，它从我的 Rails 应用程序中获取消息，并使用 socket.io-announce 模块将它们通知"到 Socket.io 房间.现在，通过使用 Socket.io 路由魔法，每个节点工作者只会向直接连接到它们的浏览器获取和发送消息.换句话说，不再有重复消息，因为 pub 和 sub 都发生在 Node.js/Socket.io 堆栈中.

So, I created a little app (with Node.js/Socket.io/Express) that took messages from my Rails app and 'announced' them into a Socket.io room using the socket.io-announce module. Now, by using Socket.io routing magic, each node worker would only get and send messages to browsers connected to them directly. In other words, no more duplicate messages since both the pub and sub happened within the Node.js/Socket.io stack.

在我清理完代码后，我会在 github 上的某个地方放一个示例.

After I get my code cleaned up I'll put an example up on a github somewhere.

这篇关于我在我的集​​群 node.js/socket.io/redis pub/sub 应用程序中收到重复的消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！