node.js多进程日志记录 [英] node.js multiprocess logging

问题描述

我现在正在基于集群的node.js项目上工作。我陷入了困境。经过研究后，我制定了解决方案。就这个。我不知道这是个好主意。这个想法是这样的。只有主进程才能写入日志文件，如果当前进程是工作进程，则它将日志消息发送给主进程，然后写入日志文件，而主进程可以直接写入日志文件。这样可以避免多个进程打开并写入同一文件。

I am now working on a node.js project based on cluster. I got stuck on the logging. After doing some research, I worked out a solution. here is it. i don't know if it is a good idea. The idea is like this. only master process can wirte to the log file, if the current process is a worker, then it send a log message to the master and then write to the log file while the master can directly write to the log file. this can avoid multiple process open and write to a same file.

var util = require('util');
var fs = require('fs');
var cluster = require('cluster');

var logger = module.exports;

var levels  = ['debug', 'info', 'warn', 'error', 'fatal'];
var logLevel = 'debug';

var logfile = null;
var errorLogfile  = null;


if(cluster.isMaster){

    logfile = fs.createWriteStream('debug.log', {flags:'a'});
    errorLogfile = fs.createWriteStream('error.log', {flags:'a'});

    cluster.on('online', function(worker){
    //collect log message from child and write to logfile.
    worker.on('message', function(msg){
        if(msg.type == 'logging') {
        var level = msg.data.level;
        var logStr = msg.data.msg;
        if(levels.indexOf(level) >= levels.indexOf('error')){
            errorLogfile.write(logStr + '\n');
        }else{
            logfile.write(logStr + '\n');
        }
        }
    });
    });
}


function log(level, args){

    if(levels.indexOf(level) < levels.indexOf(logLevel)) return;

    var args = Array.prototype.slice.call(args);

    args = args.map(function(a){
    if(typeof a !== 'string') 
        return JSON.stringify(a);
    else return a;
    });
    var msg = util.format.apply(null, args);

    var out = [];
    out.push(new Date());
    out.push('[' + level.toUpperCase() + ']');
    out.push(msg);


    if(cluster.isMaster){

    //write directly to the log file
    if(levels.indexOf(level) >= levels.indexOf('error')){
        errorLogfile.write(out.join(' ') + '\n');
    }else{
        logfile.write(out.join(' ') + '\n');
    }

    }else{

    //send to master
    cluster.worker.process.send({
        type : 'logging', 
        data : {
        level : level,
        msg : out.join(' ')
        }
    });
    }

}


logger.debug = function(){log('debug', arguments);}
logger.info = function(){log('info', arguments);}
logger.warn = function(){log('warn', arguments);}
logger.error = function(){log('error', arguments);}
logger.fatal = function(){log('fatal', arguments);}

推荐答案

1. 它必须存在瓶颈问题，因为主服务器是唯一可以记录来自以下位置的消息的地方n个工人到文件。主机与工作人员之间的通信不是必需的，因为工作人员可以直接将消息写入文件。只要消息长度小于管道缓冲区，写操作就是安全的。

1. It must have an bottleneck issue because the master is the only place who can log the messages from n workers to the file. The communication between master and workers is not necessary because workers can directly write the message to the file. As long as the message length is less than the pipe buffer, the write operation is safe.

您没有处理排水事件。当您有大量需要记录的消息时，流缓冲区将很容易装满，因为流没有足够的时间刷新并将缓冲区写入磁盘。同时，您一直将消息放入缓冲区。最后，您无法将消息完全记录到文件中。刷新缓冲区时将触发排水事件。有关排水的详细信息，请参阅 http://nodejs.org/api/stream。 html＃stream_event_drain

You did not handle "drain" events. When you have numbers of messages which needs to be logged, the stream buffer will get full easily because the stream does not have enough time to flush and write the buffer to the disk. Meanwhile, you keep putting messages to the buffer. Finally you cannot fully log the messages to the file. "drain" event will be triggered when the buffer is flushed. For the details of "drain", please refer "http://nodejs.org/api/stream.html#stream_event_drain"

PS如果您有时间，请尝试一下我的lib。它着重于多进程日志记录和日志轮换。而且它的文件记录速度非常快，内存消耗很小
https://github.com/wood1986 / ln

P.S. if you have time, please have a try my lib. It focuses on multiprocess logging and log rotation. And it is very very fast in file logging and small in memory consumption https://github.com/wood1986/ln

这篇关于node.js多进程日志记录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！