使用 mongoose 在 MongoDB 中批量更新插入 [英] Bulk upsert in MongoDB using mongoose

问题描述

是否有任何选项可以使用 mongoose 执行批量更新插入?所以基本上有一个数组，如果不存在则插入每个元素，如果存在则更新它?(我使用的是custom _ids)

Is there any option to perform bulk upserts with mongoose? So basically having an array and insert each element if it not exists or update it if it exists? (I am using customs _ids)

当我使用 .insert 时，MongoDB 会为重复键返回错误 E11000(应该更新).插入多个新文档虽然很好:

When I do use .insert MongoDB returns an error E11000 for duplicate keys (which should be updated). Inserting multiple new document works fine though:

var Users = self.db.collection('Users');

Users.insert(data, function(err){
            if (err) {
                callback(err);
            }
            else {
                callback(null);
            }
        });

使用 .save 会返回参数必须是单个文档的错误:

Using .save returns an error that the parameter must be a single document:

Users.save(data, function(err){
   ...
}

这个答案 建议没有这样的选项，但它是特定于 C# 的，并且已经有 3 年历史了.所以我想知道是否有任何选项可以使用猫鼬来做到这一点?

This answer suggest there is no such option, however it is specific for C# and also already 3 years old. So I was wondering if there is any option to do that using mongoose?

谢谢！

推荐答案

特别是在猫鼬"中，或者至少在撰写本文时还没有.MongoDB 2.6 版本的 shell 实际上使用了批量操作 API"幕后"就像所有一般的辅助方法一样.在它的实现中，它首先尝试执行此操作，如果检测到旧版本服务器，则会回退"到旧版实现.

Not in "mongoose" specifically, or at least not yet as of writing. The MongoDB shell as of the 2.6 release actually uses the "Bulk operations API" "under the hood" as it were for all of the general helper methods. In it's implementation, it tries to do this first, and if an older version server is detected then there is a "fallback" to the legacy implementation.

当前"的所有猫鼬方法都使用遗留"实现或写关注响应和基本遗留方法.但是任何给定的 mongoose 模型都有一个 .collection 访问器，它本质上访问来自底层节点本机驱动程序"的集合对象"，mongoose 是在其上实现的:

All of the mongoose methods "currently" use the "legacy" implementation or the write concern response and the basic legacy methods. But there is a .collection accessor from any given mongoose model that essentially accesses the "collection object" from the underlying "node native driver" on which mongoose is implemented itself:

 var mongoose = require('mongoose'),
     Schema = mongoose.Schema;

 mongoose.connect('mongodb://localhost/test');

 var sampleSchema  = new Schema({},{ "strict": false });

 var Sample = mongoose.model( "Sample", sampleSchema, "sample" );

 mongoose.connection.on("open", function(err,conn) { 

    var bulk = Sample.collection.initializeOrderedBulkOp();
    var counter = 0;

    // representing a long loop
    for ( var x = 0; x < 100000; x++ ) {

        bulk.find(/* some search */).upsert().updateOne(
            /* update conditions */
        });
        counter++;

        if ( counter % 1000 == 0 )
            bulk.execute(function(err,result) {             
                bulk = Sample.collection.initializeOrderedBulkOp();
            });
    }

    if ( counter % 1000 != 0 )
        bulk.execute(function(err,result) {
           // maybe do something with result
        });

 });

主要问题是猫鼬方法"实际上意识到可能尚未真正建立连接并排队"直到完成.您正在深入研究"的本机驱动程序并没有做出这种区分.

The main catch there being that "mongoose methods" are actually aware that a connection may not actually be made yet and "queue" until this is complete. The native driver you are "digging into" does not make this distinction.

所以你真的必须意识到连接是以某种方式或形式建立的.但是你可以使用本地驱动程序方法，只要你小心你在做什么.

So you really have to be aware that the connection is established in some way or form. But you can use the native driver methods as long as you are careful with what you are doing.

这篇关于使用 mongoose 在 MongoDB 中批量更新插入的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！