使用本机mongoDB驱动程序为节点流查询结果 [英] Stream query results with the native mongoDB driver for node

问题描述

我在mongoDB集合中有100,000条记录，并尝试使用本机驱动程序在node.js应用程序中检索它们.

I have 100,000 records in a mongoDB collection and trying to retrieve them in a node.js application using the native driver.

我遵循针对CursorStream的MongoDB文档但出现错误:

RangeError: Maximum call stack size exceeded

在出现此错误之前，我得到了很多这样的信息:

Before this error I get many of these:

(node) warning: Recursive process.nextTick detected. This will break in the next version of node. Please use setImmediate for recursive deferral.

这是我的代码:

var query = {...};
var fields = {...};
var options = {
    // "limit": 10
    //"skip": 10,
    //"sort": title
}

var stream = myCollection.find(query, fields, options).stream();
//  stream.pause();
var results = [];
stream.on('data', function (item){
    results.push(item);
    stream.pause();
    // Restart the stream after 1 miliscecond
    setTimeout(function() {
        stream.resume();
    }, 1);
});

stream.on('close'.....

当我没有为'data'事件定义监听器时，也会发生错误. 但是如果在创建流后立即暂停流，则不会发生这种情况.

The error occurs also when I don't define a listener for the 'data' event. but it doesn't occur if a pause the stream right after its creation.

mongod版本为v2.4.1 节点驱动程序版本为1.2.x

The mongod version is v2.4.1 The node driver version is 1.2.x

任何帮助/提示都将不胜感激.

Any help/hint would be appreciated.

推荐答案

似乎可以通过在Cursor Stream中设置批处理大小来解决问题:

It looks like the problem is solved by setting a batch size in the Cursor Stream:

var stream = myCollection.find(query, fields, options).batchSize(10000).stream();

这篇关于使用本机mongoDB驱动程序为节点流查询结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！