mongo汇总结果超出最大文档大小 [英] mongo aggregate result exceed maximum document size

问题描述

我使用mongo聚合函数在集合中查找重复的文档，其中的集合如下所示：

I use mongo aggregate function to find duplicated documents in a collection, where the collections looks like the following:

{_id, placement_id, placement_name, program_id, target}

我需要查找所有具有完全相同的文档字段_id和placement_id除外，
，因此这两个文档是相同的：

I need to find all the documents that have exactly the same fields except _id and placement_id, so this two documents are the same:

{_id:3, placement_id:23, placement_name:"pl1", program_id:5, target:"-"}
{_id:7, placement_id:55, placement_name:"pl1", program_id:5, target:"-"}

我想到的聚合函数是：

db.placements.aggregate({$group:{_id:{placement_name:"$placement_name", program_id:"$program_id", target:"$target"}, total:{$sum:1}}},{$match:{total:{$gte:2}}});

然后mongo刚返回：

Then mongo just returned:

Error: Printing Stack Trace
    at printStackTrace (src/mongo/shell/utils.js:37:15)
    at DBCollection.aggregate (src/mongo/shell/collection.js:897:9)
    at (shell):1:15
Wed Apr  2 07:43:23.090 aggregate failed: {
    "errmsg" : "exception: aggregation result exceeds maximum document size (16MB)",
    "code" : 16389,
    "ok" : 0
} at src/mongo/shell/collection.js:898

集合是正确的，我在一个较小的集合上对其进行了测试，并且效果很好，但是生产集合包含大约8000万个文档。我想知道当在80M个文档上尝试使用find（）函数时，该函数是否有效，并要求您键入 it以获取更多记录。汇总功能为什么没有此功能？我也尝试将limit（）附加到聚合函数的末尾，但是它也不起作用。有没有解决的办法？谢谢。

the aggregate is correct, I tested it on a smaller collection and it works fine, but the production collection has about 80M documents. I was wondering when trying the find() function on 80M documents, it works and asking you to type 'it' for more records. How come the aggregate function doesn't have this capability? I also tried to append limit() to the end of the aggregate function but it won't work either. Any work around? Thanks.

推荐答案

聚合函数为什么没有此功能？

how come aggregate function don't have this capability?

它在2.6中会很快推出。

It does in 2.6 which will be out very soon.

i还尝试将limit（）附加到聚合函数的末尾，它也不起作用。

i also tried to append limit() to the end of the aggregate function, it won't work either. any work around?

聚合框架中有一个 $ limit 运算符： http://docs.mongodb.org/manual/reference/operator/aggregation/limit/

There is a $limit operator for the aggregation framework: http://docs.mongodb.org/manual/reference/operator/aggregation/limit/

这篇关于mongo汇总结果超出最大文档大小的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！