如何在不超过最大文档大小的情况下编写聚合? [英] How could I write aggregation without exceeds maximum document size?

问题描述

通过查询，我得到了exceeds maximum document size problem异常，

I got exceeds maximum document size problem exception by the query as follows,

pipe = [
    {"$match": { "birthday":{"$gte":datetime.datetime(1987, 1, 1, 0, 0)} }}
    ]
res =db.patients.aggregate(pipe,allowDiskUse=True)

我通过添加$project运算符

但是，即使我仍使用$project，文档仍在16MB上怎么办?

However what if the document still over 16MB even I use $project ?

我该怎么办?任何的想法 ?谢谢

What can I do ? any idea ? Thank you

pipe = [
    {"$project": {"birthday":1, "id":1}
    },
    {"$match": { "birthday":{"$gte":datetime.datetime(1987, 1, 1, 0, 0)} }
     }
    ]
res =db.patients.aggregate(pipe,allowDiskUse=True)

异常

OperationFailure: command SON([('aggregate', 'patients'), ('pipeline', [{'$match': {'birthday': {'$gte': datetime.datetime(1987, 1, 1, 0, 0)}}}]), ('allowDiskUse', True)]) on namespace tw_insurance_security_development.$cmd failed: exception: aggregation result exceeds maximum document size (16MB)

推荐答案

默认情况下，聚合结果将在单个BSON文档中返回给您，这是大小限制的来源.如果您需要返回的收益还不止这些，则可以:

By default the result of aggregations are returned to you in a single BSON document, which is where the size restriction comes from. If you need to return more than that you can either:

• 已将结果输出到集合.您可以通过使用

• have the results be output to a collection. You do this by finishing your pipeline with

{"$ out":某些集合名称"}

{"$out": "some-collection-name"}

然后，您可以正常查询该集合(使用完后需要将其删除)

You then query that collection as normal (you'll need to delete it yourself when you're done with it)

具有作为游标返回的结果.

have the results returned as a cursor, by specifying useCursor=True when you call aggregate.

这两个选项都需要mongodb 2.6:如果您仍在运行mongodb 2.4，那么这只是聚合的基本限制.

Both of these options require mongodb 2.6: if you are still running mongodb 2.4 then this is just a fundamental limit of aggregations.

这篇关于如何在不超过最大文档大小的情况下编写聚合?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！