在使用JSON.stringify的Node.js中导致“进程内存不足"错误 [英] In Node.js using JSON.stringify results in 'process out of memory' error

问题描述

使用Node我试图从LDAP服务器收集用户数据，然后将该数据写入JSON文件.我正在使用以下代码执行此操作:

With Node I am trying to collect user data from an LDAP server and then write that data to a JSON file. I am using the following code to do this:

fs.writeFile('data.json', JSON.stringify(data, null, 4));

问题是JSON.stringify方法导致以下错误:

The problem is the JSON.stringify method is causing the following error:

FATAL ERROR: JS Allocation failed - process out of memory

我知道问题出在JSON.stringify上，因为如果我使用console.log而不是fs.writeFile，则会出现相同的错误.

I know the problem is with JSON.stringify because if I use console.log rather than fs.writeFile I get the same error.

我正在尝试写入大量数据(LDAP数据库中超过500个条目).有谁知道我该怎么做?这是完整的代码:

I am trying to write a lot of data (over 500 entries in the LDAP database). Does anyone know how I can get this to work? Here is the code in full:

var ldap = require('ldapjs');
var util = require('util');
var fs = require('fs');
var client = ldap.createClient({
  url: '************'
});

client.bind('CN=**********,OU=Users,OU=LBi UK,OU=UK,DC=********,DC=local', '*********', function(err) {
  if (err) {
    console.log(err.name);
  }
});


// taken from http://ldapjs.org/client.html
client.search('OU=Users,OU=******,OU=UK,DC=******,DC=local', {
  scope: 'sub',
  filter: 'objectClass=organizationalPerson',
  attributes: ['givenName', 'dn', 'sn', 'title', 'department', 'thumbnailPhoto', 'manager']
  // filter by organizational person
}, function(err, res) {
  if (err) {
    console.log(err.name);
  }

  var limit = 1;
  var data = {"directory": []};

  res.on('searchEntry', function(entry) {

    var obj = {};
    entry.attributes.forEach(function (attribute) {
      var value;
      if (attribute.type === 'thumbnailPhoto') {
        value = attribute.buffers[0];

      } else {
        value = attribute.vals[0];
      }
      obj[attribute.type] = value;
    });
    data.directory.push(obj);
  });
  res.on('error', function(err) {
    console.log('error: ' + err.message);
  });
  res.on('end', function(result) {
    fs.writeFile('data.json', JSON.stringify(data, null, 4));
  });

});

推荐答案

正如@freakish提到的那样，问题在于我的数据太大.

As @freakish mentioned the problem was my data was too big.

之所以如此之大，是因为有大量图像作为对象返回.最后，我要做的就是使用Buffers将对象编码为base64，然后数据的大小变得更易于管理.

The reason the data was so big was due to a large number of images that were being returned as objects. In the end all I needed to do was encode the object as base64 using Buffers and then the size of the data became much more manageable.

这篇关于在使用JSON.stringify的Node.js中导致“进程内存不足"错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！