从nodejs在mongodb中插入大量对象 [英] Inserting a big array of object in mongodb from nodejs

问题描述

我需要从nodejs在mongodb中插入一大堆对象(大约1.5-2百万).如何改善插入效果?

I need to insert a big array of objects (about 1.5-2 millions) in mongodb from nodejs. How can i improve my inserting?

这是我的代码:

var sizeOfArray = arrayOfObjects.length; //sizeOfArray about 1.5-2 millions
for(var i = 0; i < sizeOfResult; ++i) {
  newKey = {
    field_1: result[i][1],
    field_2: result[i][2],
    field_3: result[i][3]
  };
  collection.insert(newKey, function(err, data) {
    if (err) {
      log.error('Error insert: ' + err);
    }
  });
}

推荐答案

您可以使用有两种类型的批量操作:

There are two types of bulk operations:

1. 已订购的批量操作.这些操作按顺序执行所有操作，并在第一个写入错误时出错.
2. 无序批量操作.这些操作并行执行所有操作，并汇总所有错误.无序 批量操作不能保证执行顺序.

1. Ordered bulk operations. These operations execute all the operation in order and error out on the first write error.
2. Unordered bulk operations. These operations execute all the operations in parallel and aggregates up all the errors. Unordered bulk operations do not guarantee order of execution.

因此您可以执行以下操作:

So you can do something like this:

var MongoClient = require('mongodb').MongoClient;

MongoClient.connect("mongodb://myserver:27017/test", function(err, db) {
    // Get the collection
    var col = db.collection('myColl');

    // Initialize the Ordered Batch
    // You can use initializeUnorderedBulkOp to initialize Unordered Batch
    var batch = col.initializeOrderedBulkOp();

    for (var i = 0; i < sizeOfResult; ++i) {
      var newKey = {
          field_1: result[i][1],
          field_2: result[i][2],
          field_3: result[i][3]
      };
      batch.insert(newKey);
    }

    // Execute the operations
    batch.execute(function(err, result) {
      console.dir(err);
      console.dir(result);
      db.close();
    });
});

这篇关于从nodejs在mongodb中插入大量对象的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！