nodejs http response.write:是否可能内存不足? [英] nodejs http response.write: is it possible out-of-memory?

问题描述

如果我有以下代码每隔10ms重复向客户端发送一次数据:

If i have following code to send data repeatedly to client every 10ms:

setInterval(function() {
    res.write(somedata);
}, 10ms);

如果客户端接收数据的速度非常慢，会发生什么?

What would happen if the client is very slow to receive the data?

服务器会出现内存不足错误吗?

Will server get out-of-memory error?

实际上保持连接有效，服务器不断发送jpeg数据(HTTP multipart/x-mixed-replace标头+正文+头+正文.....)
由于node.js response.write是异步的，因此
因此一些用户猜测它可能会将数据存储在内部缓冲区中，然后等到低层告知它可以发送时，
这样内部缓冲区会增加，对吗?

如果我是对的，那么如何解决呢?
问题是在为单个写调用发送数据时，node.js不会通知我.

换句话说，我无法告诉用户这种方式在理论上没有内存不足"以及如何解决的风险.

actually the connection is kept alive, sever send jpeg data endlessly (HTTP multipart/x-mixed-replace header + body + header + body.....)
Because node.js response.write is asynchronous,
so some users guess it may store data in internal buffer and wait until low layer tells it can send,
so the internal buffer will grow, am i right?

If i am right, then how to resolve this?
the problem is node.js does not notify me when data is send for a single write call.

In other word, i can not tell user this way is theoretically no risk of "out of memory" and how to fix it.

更新:通过user568109给出的关键字"drain"事件，我研究了node.js的来源，并得出结论:
这确实会导致内存不足"错误.我应该检查response.write(...)=== false的返回值，然后处理响应的排水"事件.

Update: By the keyword "drain" event given by user568109, i studied the source of node.js, and got conclusion:
it really will cause "out-of-memory" error. I should check return value of response.write(...)===false and then handle "drain" event of the response.

http.js:

OutgoingMessage.prototype._buffer = function(data, encoding) {
  this.output.push(data); //-------------No check here, will cause "out-of-memory"
  this.outputEncodings.push(encoding);

  return false;
};


OutgoingMessage.prototype._writeRaw = function(data, encoding) { //this will be called by resonse.write
  if (data.length === 0) {
    return true;
  }

  if (this.connection &&
      this.connection._httpMessage === this &&
      this.connection.writable &&
      !this.connection.destroyed) {
    // There might be pending data in the this.output buffer.
    while (this.output.length) {
      if (!this.connection.writable) {    //when not ready to send
        this._buffer(data, encoding);    //----------> save data into internal buffer
        return false;
      }
      var c = this.output.shift();
      var e = this.outputEncodings.shift();
      this.connection.write(c, e);
    }

    // Directly write to socket.
    return this.connection.write(data, encoding);
  } else if (this.connection && this.connection.destroyed) {
    // The socket was destroyed.  If we're still trying to write to it,
    // then we haven't gotten the 'close' event yet.
    return false;
  } else {
    // buffer, as long as we're not destroyed.
    this._buffer(data, encoding);
    return false;
  }
};

推荐答案

一些陷阱:

1. 如果通过http发送，则不是一个好主意.如果请求未在指定的时间内完成，则浏览器可能会将其视为超时.服务器也将关闭空闲时间过长的连接.如果客户无法跟上，超时几乎可以确定.

1. If sending over http it is not be a good idea. The browser may consider the request as timeout if it is not finished within specified amount of time. Server too will close connection which is idle for too long. If client cannot keep up, the timeout is almost certain.

setInterval 10ms也受到一些限制.这并不意味着它将每隔10ms重复一次，而10ms是等待重复的最小时间.这将比您设置的间隔慢.

setInterval for 10ms is also subject to some restrictions. It doesn't mean it will repeat after every 10ms, 10ms is the minimum it will wait before repeating. It will be slower than what you set the interval.

假设您有机会使响应过载数据，然后在某个时候服务器将终止连接并根据设置的限制通过 413 Request Entity Too Large 进行响应.

Let's say you chance to overload the response with data, then at some point the server will end connection and respond by 413 Request Entity Too Large depending on what the limit is set.

Node.js具有单线程体系结构，最大内存限制约为1.7 GB.如果您 设置 ，您的上述服务器限制过高，并且有许多传入连接，则会出现进程内存不足错误.

Node.js has single threaded architecture with a max memory limitation of around 1.7 GB. If you set your above server limits to too high and have many incoming connections you will get process out of memory error.

因此，如果有适当的限制，它将导致超时或请求太大.(而且程序中没有其他错误.)

So with appropriate limits it will either give timeout or be request too large. (And there are no other errors in your program.)

更新

您需要使用 drain 事件.http响应是可写流.它具有自己的内部缓冲区.清空缓冲区后，将触发耗尽事件.您应该更深入地了解有关流的更多信息.这将不仅在http中提供帮助.您可以找到有关网络流的几种资源.

You need to use drain event. The http response is a writable stream. It has its own internal buffer. When the buffer is emptied the drain event is triggered. You should learn more about streams as you would go in deeper. This will help you not just in http. You can find several resources about streams on web.

这篇关于nodejs http response.write:是否可能内存不足?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！