Node.JS + mongo：.find（）。each（）在第一批后停止 [英] Node.JS + mongo: .find().each() stopping after first batch

问题描述

这让我很难过。

我有一个独立的（命令行执行的）节点脚本，其目的是遍历大集合中的所有文档（其中有几十万个），对于每个文档，执行一些计算，运行一些额外的JS代码，然后使用一些新值更新文档。

Per 的文档 cursor.each（），一旦我从 collection.find（）获得光标， .each（ cb）方法应该对整个集合中的每个项目执行 cb（item）。

示例代码：

  myDb.collection（'bigcollection'）。find（）。each（function（err，doc） {
 if（err）{
 console.log（Error：+ err）; 
} else {
 if（doc！= null）{
 process .stdout.write（。）; 
} else {
 process.stdout.write（X）; 
} 
} 
}）; 
  

我期望这样做是打印出数十万。 然后在最后打印 X ，如 cursor.each（）应该迭代此游标的所有文档，并根据示例代码，如果该项为null，则光标用尽/空并关闭。

<但是实际所做的是打印出精确的101 。，没有 X 最后。

如果我调整批量大小（ .find（）。batchSize（10）.each（... ），它会在捞出之前完成相同数量的文件。

那么，为什么它只处理第一批？我在某种程度上误读了.each（）的文档？它是否与这是一个命令行脚本有关，并且在第二批结果返回之前，某个整个脚本正在退出，或者其他什么？如果是这样，我该怎么做确定它实际上处理所有resu lts？

作为一个副节点，我尝试过使用.stream（）和.forEach（），在这两种情况下，它都在第一批。

更新：
嗯，这很有趣。刚尝试连接到我的生产服务器而不是我在localhost上的mongo实例，瞧，它会像它应该的那样贯穿整个集合。服务器正在运行mongodb 3.0.6，我的本地实例是3.2.3。我的节点mongodb驱动程序的版本是2.0.43。

解决方案

我的集合中有200个文档，代码顺利。换句话说，无法重现问题。如您所见，我已将批量减小到10个。

  var url ='mongodb：// localhost：27017 / test' ; 
 MongoClient.connect（url，function（err，db）{
 if（err）{
 console.log（err）; 
} 
 else {
 var counter = 0; 
 db.collection（'collection'）。find（{}）。batchSize（10）.each（function（e，r）{
 if（err）{
 console.log（E：+ err）; 
 db.close（）; 
} 
 else {
 if（r == null）{
 db.close（）; 
} 
 else {
 counter + = 1; 
 console.log（X：+ counter）; 
} 
} 
}）; 
} 
}）; 
  

如果您仍然遇到同样的问题，我建议将MongoDB驱动程序更新到最新版本。由于驱动程序正在积极开发，有时bug会潜入已发布的版本，导致奇怪的行为。

This has me stumped.

I have a standalone (command-line executed) node script, whose purpose is to iterate through all the documents in a large collection (several hundred thousand of them), and for each document, perform a few calculations, run a little additional JS code, and then update the document with some new values.

Per the documentation for cursor.each(), once I've got my cursor from collection.find(), the .each(cb) method should execute cb(item) on each item in the entire collection.

Example code:

myDb.collection('bigcollection').find().each(function(err, doc) {
    if (err) {
        console.log("Error: " + err);
    } else {
        if (doc != null) {
            process.stdout.write(".");
        } else {
            process.stdout.write("X");
        }
    }
});

What I'd expect this to do is print out several hundred thousand .'s and then print an X at the end, as cursor.each() is supposed to "Iterate over all the documents for this cursor," and per the example code, "If the item is null then the cursor is exhausted/empty and closed."

But what it actually does is print out precisely 101 .'s, without an X at the end.

If I adjust the batch size (.find().batchSize(10).each(...), it goes through exactly that number of documents before bailing.

So, why is it only processing the first batch? Am I somehow misreading the documentation for .each()? Does it have to do with the fact that this is a command-line script, and somehow the whole script is exiting before the second batch of results comes back, or something? If so, how do I make sure it actually processes all the results?

As a side node, I've tried using .stream() and .forEach(), and in both of those cases as well, it ditches after the first batch.

UPDATE: Well, this is interesting. Just tried connecting to my production server instead of my mongo instance on localhost, and voila, it runs through the entire collection like it should. The server is running mongodb 3.0.6, my local instance is 3.2.3. My version of the node mongodb driver is 2.0.43.

解决方案

I have 200 documents in my collection and following code goes well. In other words, couldn't reproduce problem. As you can see I have reduced batch size to 10.

var url = 'mongodb://localhost:27017/test';
MongoClient.connect(url, function(err, db) {
    if (err) {
        console.log(err);
    }
    else {
        var counter = 0;
        db.collection('collection').find({}).batchSize(10).each(function(e, r){
            if(err){
                console.log("E: " +  err);
                db.close();
            }
            else{
                if(r ==  null){
                    db.close();
                }
                else{
                counter += 1;
                console.log("X: " +  counter);
                }
            }
        });
    }
});

If you are still facing same issue, I'd suggest to update MongoDB driver to latest version. Since drivers are actively being developed, sometime bugs sneak into released version causing strange behavior.

这篇关于Node.JS + mongo：.find（）。each（）在第一批后停止的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！