与Node.js的长连接，如何减少内存使用，防止内存泄漏?还与 V8 和 webkit-devtools 相关 [英] Long connections with Node.js, how to reduce memory usage and prevent memory leak? Also related with V8 and webkit-devtools

问题描述

这是我正在尝试做的事情:我正在开发一个 Node.js http 服务器，它将在一台机器上从数万个移动客户端保持长连接以用于推送目的(与 redis 协作).

测试环境:

1.80GHz*2 CPU/2GB RAM/Unbuntu12.04/Node.js 0.8.16

我第一次使用express"模块，在使用swap之前我可以达到大约120k的并发连接，这意味着RAM不够用.然后，我切换到原生的http"模块，我得到了大约 160k 的并发.但是我意识到原生http模块还有太多我不需要的功能，所以我把它切换到原生的net"模块(这意味着我需要自己处理http协议，但这没关系).现在，我可以达到每台机器大约 25 万个并发连接.

这是我的代码的主要结构:

var net = require('net');var redis = require('redis');var pendingClients = {};var redisClient = redis.createClient(26379, 'localhost');redisClient.on('message', function (channel, message) {var client = pendingClients[channel];如果(客户端){client.res.write(message);}});var server = net.createServer(函数(套接字){var 缓冲区 = '';socket.setEncoding('utf-8');socket.on('数据', onData);函数 onData(块){缓冲区 += 块；//解析请求数据.//...if ('我已经得到了我需要的一切') {socket.removeListener('data', onData);变量请求 = {客户 ID:'随便'};var res = new ServerResponse(socket);server.emit('request', req, res);}}});server.on('request', function (req, res) {如果(res.socket.destroyed){返回;}待定Clinets[req.clientId] = {资源:资源};redisClient.subscribe(req.clientId);res.socket.on('错误'，函数(错误){控制台日志(错误)；});res.socket.on('close', function () {删除pendingClients[req.clientId];redisClient.unsubscribe(req.clientId);});});服务器.听(3000)；功能服务器响应(套接字){this.socket = 套接字；}ServerResponse.prototype.write = 函数(数据){this.socket.write(data);}

最后，我的问题是:

1. 如何减少内存使用量以进一步提高并发性?

2. 我真的很困惑如何计算 Node.js 进程的内存使用情况.我知道由 Chrome V8 提供支持的 Node.js，有 process.memoryUsage() api，它返回三个值:rss/heapTotal/heapUsed，它们之间有什么区别，我应该更关注哪一部分，Node.js 进程使用的内存的确切组成是什么?

3. 我担心内存泄漏，尽管我做了一些测试并且似乎没有问题.有什么我应该关注的地方或任何建议吗?

4. 我找到了关于 .据我所知，堆是 unix 系统中动态分配内存的来源.因此，堆总数似乎是在堆上为您的使用分配的所有内容，而使用的堆是您已使用的分配量.

5. 你的内存使用情况很好，看起来你实际上没有泄漏.我还不会担心.=]

6. 不知道.

7. 这个快照看起来很合理.我预计从大量请求中创建的一些对象已经被垃圾收集了，而另一些则没有.您会看到没有超过 10k 的对象，而且这些对象中的大多数都非常小.我称之为好.

不过，更重要的是，我想知道您是如何对此进行负载测试的.我以前尝试过像这样进行大规模的负载测试，由于打开文件描述符的数量(通常每个进程大约一千个)的限制，大多数工具根本无法在 linux 上生成这种负载).同样，一旦使用了套接字，就不能立即再次使用它.我记得，它需要很短的一分钟时间才能再次使用.在此与我通常看到系统范围打开的文件描述符限制设置在 100k 以下这一事实之间，我不确定是否有可能在未修改的盒子上接收那么多负载，或者在单个盒子上生成它.由于您没有提到任何此类步骤，我认为您可能还需要调查您的负载测试，以确保它按照您的想法进行操作.

Here is what I'm trying to do: I'm developing a Node.js http server, which will hold long connections for pushing purpose(collaborate with redis) from tens of thousands of mobile clients in a single machine.

Test environment：

1.80GHz*2 CPU/2GB RAM/Unbuntu12.04/Node.js 0.8.16

At the first time, I used "express" module, with which I could reach about 120k concurrent connections before swap being used which means the RAM is not enough. Then, I switched to native "http" module, I got the concurrency up to about 160k. But I realized that there are still too many functionality I don't need in native http module, so I switched it to native "net" module(this means I need to handle http protocol by myself, but that's ok). now, I can reach about 250k concurrent connections per single machine.

Here is the main structure of my codes:

var net = require('net');
var redis = require('redis');

var pendingClients = {};

var redisClient = redis.createClient(26379, 'localhost');
redisClient.on('message', function (channel, message) {
    var client = pendingClients[channel];
    if (client) {
        client.res.write(message);
    }
});

var server = net.createServer(function (socket) {
    var buffer = '';
    socket.setEncoding('utf-8');
    socket.on('data', onData);

    function onData(chunk) {
        buffer += chunk;
        // Parse request data.
        // ...

        if ('I have got all I need') {
            socket.removeListener('data', onData);

            var req = {
                clientId: 'whatever'
            };
            var res = new ServerResponse(socket);
            server.emit('request', req, res);
        }  
    }
});

server.on('request', function (req, res) {
    if (res.socket.destroyed) {            
        return;
    }

    pendingClinets[req.clientId] = {
        res: res
    };

    redisClient.subscribe(req.clientId);

    res.socket.on('error', function (err) {
        console.log(err);
    });

    res.socket.on('close', function () {
        delete pendingClients[req.clientId];

        redisClient.unsubscribe(req.clientId);
    });
});

server.listen(3000);

function ServerResponse(socket) {
    this.socket = socket;
}
ServerResponse.prototype.write = function(data) {
    this.socket.write(data);
}

Finally, here are my questions:

1. How can I reduce the memory usage so that increase the concurrency farther?

2. I'm really confused about how to calculate the memory usage of Node.js process. I know Node.js powered by Chrome V8, there is process.memoryUsage() api and it return three values: rss/heapTotal/heapUsed, what's the difference between them, which part should I concern more, and what's the exactly composition of the memory used by the Node.js process?

3. I worried about memory leak even though I have done some tests and there don't seem to be a problem. Are there any points I should concern or any advises?

4. I found a doc about V8 hidden class, as it described, does that mean whenever I add a property named by clientId to my global object pendingClients just like my codes above, there will be a new hidden class be generated? Dose it will cause memory leak?

5. I used webkit-devtools-agent to analyze heap map of the Node.js process. I started the process and took a heap snapshot, then I sent 10k requests to it and disconnected them later, after that I took a heap snapshot again. I used the comparison perspective to see the difference between these two snapshots. Here is what I got: Could anyone explain this? The number and size of (array)/(compiled code)/(string)/Command/Array increased a lot, what does this mean?

EDIT: How did I run the loading test?
1. Firstly, I modified some parameters both on server machine and client machines（to achieve more than 60k concurrency need more than one client machine, because one machine only have 60k+ ports(represented by 16 bit) at most）
1.1. Both one the server and the client machines, I modified the file descriptor use these commands in the shell where the test program will be run in:

ulimit -Hn 999999
ulimit -Sn 999999

1.2. On the server machine, I also modified some net/tcp related kernel parameters, the most important ones are:

net.ipv4.tcp_mem = 786432 1048576 26777216
net.ipv4.tcp_rmem = 4096 16384 33554432
net.ipv4.tcp_wmem = 4096 16384 33554432

1.3. As to the client machines:

net.ipv4.ip_local_port_range = 1024 65535

2. Secondly, I wrote a custom simulate client program using Node.js, since most load test tools, ab, siege, etc, are for short connections, but I'm using long connections and have some special requirements.
3. Then I started the server program on a single machine, and three client program on the other three separated machines.

EDIT: I did reach 250k concurrent connections on a single machine(2GB RAM), but turned out, it's not very meaningful and practical. Because when a connection connected, I just let the connection pending, nothing else. When I tried to sent response to them, the concurrency number dropped down to 150k around. As I calculated, there is about 4KB more memory usage per connection, I guess it's related to net.ipv4.tcp_wmem which I set to 4096 16384 33554432, but even I modified it to smaller, nothing changed. I can't figure out why.

EDIT: Actually, now I'm more interested in how much memory per tcp connection uses and what's the exactly composition of the memory used by a single connection? According to my test data:

150k concurrency consumed about 1800M RAM(from free -m output), and the Node.js process had about 600M RSS

Then, I assumed this:

• (1800M - 600M) / 150k = 8k, this is the kernel TCP stack memory usage of a single connection, it consists of two parts: read buffer(4KB) + write buffer(4KB)(Actually, this doesn't match my setting of net.ipv4.tcp_rmem and net.ipv4.tcp_wmem above, how does the system determine how much memory to use for these buffers?)

• 600M / 150k = 4k, this is the Node.js memory usage of a single connection

Am I right? How can I reduce the memory usage in both aspects?

If there are anywhere I didn't describe well, let me know, I'll refine it! Any explanations or advises will be appreciated, thanks!

解决方案

1. I think you shouldn't worry about further decreasing memory usage. From that readout you included, it seems you're pretty close to the bare minimum conceivable (I interpret it as being in bytes, which is standard when a unit isn't specified).

2. This is a more in depth question than I can answer, but here's what RSS. The heap is where dynamically allocated memory comes from in unix systems, as best I understand. So, the heap total seems like it'd be all that is allocated on the heap for your usage, whereas the heap used is how much of what's allocated you've used.

3. Your memory usage is quite good, and it doesn't seem you actually have a leak. I wouldn't worry yet. =]

4. Don't know.

5. This snapshot seems reasonable. I expect some of the objects created from the surge of requests had been garbage collected, and others hadn't. You see there's nothing over 10k objects, and most of these objects are quite small. I call that good.

More importantly, though, I wonder how you're load testing this. I've tried to do massive load testing like this before, and most tools simply can't manage to generate that kind of load on linux, because of the limits on the number of open file descriptors (generally around a thousand per process by default). As well, once a socket is used, it is not immediately available for use again. It takes some significant fraction of a minute, as I recall, to be usable again. Between this and the fact that I've normally seen the system wide open file descriptor limit set somewhere under 100k, I'm not sure it's possible to receive that much load on an unmodified box, or to generate it on a single box. Since you don't mention any such steps, I think you might also need to investigate your load testing, to make sure it's doing what you think.

这篇关于与Node.js的长连接，如何减少内存使用，防止内存泄漏?还与 V8 和 webkit-devtools 相关的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！