有数百万条记录时，Mongo计数真的很慢 [英] Mongo count really slow when there are millions of records

问题描述

//FAST
db.datasources.find().count()
12036788

//SLOW    
db.datasources.find({nid:19882}).count()
10161684

索引索引

有什么方法可以使第二个查询更快? (大约需要8秒钟)

解决方案

由于MongoDB仍然必须进行完整的b树遍历才能找到适当数量的文档，因此索引或其他方式的查询计数很慢符合您的条件.这样做的原因是，MongoDB b树结构未计数"，这意味着每个节点都不存储有关节点/子树中元素数量的信息.

此问题在这里报告 https://jira.mongodb.org/browse/SERVER-1752 ，除了手动维护该集合的计数器(显然存在一些缺点)之外，目前没有其他方法可以提高性能.

还请注意，db.col.count()版本(因此没有条件)可以采用很大的快捷方式，并且实际上不执行查询，因此速度很高.也就是说，它并不总是报告与应该返回所有元素的计数查询相同的值(例如，不会在具有高写吞吐量的分片环境中使用).讨论是否是错误.我认为是.

请注意，在2.3及更高版本中，引入了重大优化，该优化应该(并且确实)提高索引字段上计数的性能.请参阅: https://jira.mongodb.org/browse/SERVER-7745

//FAST
db.datasources.find().count()
12036788

//SLOW    
db.datasources.find({nid:19882}).count()
10161684

Index on nid

Any way to make the second query faster? (It is taking about 8 seconds)

解决方案

Count queries, indexed or otherwise, are slow due to the fact that MongoDB still has to do a full b-tree walk to find the appropriate number of documents that match your criteria. The reason for this is that the MongoDB b-tree structure is not "counted" meaning each node does not store information about the amount of elements in the node/subtree.

The issue is reported here https://jira.mongodb.org/browse/SERVER-1752 and there is currently no workaround to improve performance other than manually maintaining a counter for that collection which obviously comes with a few downsides.

Also note that the db.col.count() version (so no criteria) can take a big shortcut and doesn't actually perform a query hence it's speed. That said it does not always report the same value as a count query would that should return all elements (it won't be in sharded environments with high write throughput for example). Up for debate whether or not that's a bug. I think it is.

Note that in 2.3+ a significant optimization was introduced that should (and does) improve performance of counts on indexed fields. See : https://jira.mongodb.org/browse/SERVER-7745

这篇关于有数百万条记录时，Mongo计数真的很慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！