Node.js中的猫鼬无声崩溃 [英] Mongoose silent crash in Node.js

问题描述

我试图在mongodb中插入大量普通对象，但是正在发生一些非常奇怪的事情.假设我尝试插入1000个这样的对象

I'm trying to insert a bulk of plain objects to mongodb, but something very weird is happening. Let's say i try to insert 1000 objects like this

for (var i = 0; i < 1000; i++) {
     var commentAux = {
       'a': 'test' + i,
       'b': 'https://test.com',
       'c': appUser.id,
       'd': 'test of' + i
     }
     comments.push(commentAux);
}
Comment.collection.insert(comments, function (err, docs) {
   if (err) {
     return res.json({success: false, message: 'Comments Fail'});
   }
}

但是，如果我将1000更改为1500，服务器将挂起.它从不抛出惊吓，也没有警告，什么也没有.它只是卡在那里.我一直在阅读，而且文档数量甚至还没有达到mongo支持的限制数量.

But if I change the 1000 for lets say 1500, the server hangs. It never throws an excetpion, neither a warning, nothing. It just stucks there. I've been reading, and this amount of documents isn't even near the limit amount that mongo supports.

有人遇到同样的问题吗?我在Windows中使用mongo 3.2

Someone face the same issue? I'm using mongo 3.2 in windows

推荐答案

如评论中所述，批量最大大小为1000 ...所以我实现了一个自定义的BulkManager来管理我的大量文档API.

As mentioned in the comments, the bulk max size is 1000...so i've implemented a custom BulkManager in order to manage high amounts of documents in my API.

所以我的BulkManager.js就是这个.

So my BulkManager.js it this.

module.exports = function () {
    var instance = {};
    var Promise = require("bluebird");
    var Q = require("q");
    Promise.promisifyAll(instance);

    instance.insert = function (DbModel, items) {
        var arrays = [], size = 1000;

        while (items.length > 0)
            arrays.push(items.splice(0, size));

        return saveItems(DbModel, arrays, arrays.length, 0);
    };

    var deferred = Q.defer();

    function saveItems(DbModel, arrays, amountToProcess, indexToProcess) {
        if (indexToProcess <= (amountToProcess - 1)) {
            var items = arrays[indexToProcess];
            DbModel.collection.insert(items, function (err, docs) {
                if (err) {
                    return deferred.reject({success: false, error: err});
                }

                if (amountToProcess > 1 && indexToProcess < (amountToProcess - 1)) {
                    indexToProcess++;
                    saveItems(DbModel, arrays, amountToProcess, indexToProcess);
                } else {
                    return deferred.resolve({success: true, docs: docs});
                }
            });
        }
        return deferred.promise;
    }
    return instance;
};

从我的API中，我这样称呼它.

And from my API I call it like this.

BulkManager.insert(PhotoModel, photos).then(function (response) {
    if (!response.success) {
        return res.json({success: false, message: 'Photos Fail'});
    }
    // do stuff on success
}

这篇关于Node.js中的猫鼬无声崩溃的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！