如何在 Mongoskin 中使用无序批量插入? [英] How to use unordered bulk inserting with Mongoskin?

问题描述

我在使用 Mongoskin 在 Node 上执行批量插入(MongoDB 2.6+)时遇到问题.

I'm having trouble using Mongoskin to perform bulk inserting (MongoDB 2.6+) on Node.

var dbURI = urigoeshere;
var db = mongo.db(dbURI, {safe:true});
var bulk = db.collection('collection').initializeUnorderedBulkOp();

for (var i = 0; i < 200000; i++) {
    bulk.insert({number: i}, function() {
        console.log('bulk inserting: ', i);
    });
}

bulk.execute(function(err, result) {
    res.json('send response statement');
}); 

上面的代码给出了以下警告/错误:

The above code gives the following warnings/errors:

(node) warning: possible EventEmitter memory leak detected. 51 listeners added. Use emitter.setMaxListeners() to increase limit.
TypeError: Object #<SkinClass> has no method 'execute'
(node) warning: possible EventEmitter memory leak detected. 51 listeners added. Use emitter.setMaxListeners() to increase limit.
TypeError: Object #<SkinClass> has no method 'execute'

是否可以使用 Mongoskin 进行无序批量操作?如果是这样，我做错了什么?

Is it possible to use Mongoskin to perform unordered bulk operations? If so, what am I doing wrong?

推荐答案

您可以这样做，但您需要更改调用约定来执行此操作，因为只有回调"表单才会实际返回一个集合对象，可以从中调用 .initializeUnorderedBulkOp() 方法.您认为它的工作方式也存在一些用法差异:

You can do it but you need to change your calling conventions to do this as only the "callback" form will actually return a collection object from which the .initializeUnorderedBulkOp() method can be called. There are also some usage differences to how you think this works:

var dbURI = urigoeshere;
var db = mongo.db(dbURI, {safe:true});
db.collection('collection',function(err,collection) {
    var bulk = collection.initializeUnorderedBulkOp();
    count = 0;

    for (var i = 0; i < 200000; i++) {
        bulk.insert({number: i});
        count++;

        if ( count % 1000 == 0 )
            bulk.execute(function(err,result) {
               // maybe do something with results
               bulk = collection.initializeUnorderedBulkOp(); // reset after execute
            });      

    });

    // If your loop was not a round divisor of 1000
    if ( count % 1000 != 0 )
        bulk.execute(function(err,result) {
          // maybe do something here
        });
});

因此实际的批量"方法本身不需要回调，并且完全按照 文档.异常是 .execute() ，它实际上将语句发送到服务器.

So the actual "Bulk" methods themselves don't require callbacks and work exactly as shown in the documentation. The exeception is .execute() which actually sends the statements to the server.

虽然驱动程序会为您解决一些问题，但在调用 execute 之前将太多操作排队可能不是一个好主意.这基本上是在内存中建立的，尽管驱动程序一次只会发送 1000 个批次(这是服务器限制以及完整批次低于 16MB)，但您可能需要在这里进行更多控制，至少限制内存使用.

While the driver will sort this out for you somewhat, it probably is not a great idea to queue up too many operations before calling execute. This basically builds up in memory, and though the driver will only send in batches of 1000 at a time ( this is a server limit as well as the complete batch being under 16MB ), you probably want a little more control here, at least to limit memory usage.

如图所示，这就是模测试的重点，但是如果用于构建操作的内存和可能非常大的响应对象对您来说不是问题，那么您可以继续排队操作并调用 .execute() 一次.

That is the point of the modulo tests as shown, but if memory for building the operations and a possibly really large response object are not a problem for you then you can just keep queuing up operations and call .execute() once.

响应"的格式与BulkWriteResult<的文档中给出的格式相同/a>.

The "response" is in the same format as given in the documentation for BulkWriteResult.

这篇关于如何在 Mongoskin 中使用无序批量插入?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！