如何有效地执行“与众不同"?有多个键? [英] How to efficiently perform "distinct" with multiple keys?

问题描述

例如，有一个像这样的集合:

For example, there is a collection like this:

{市场:'SH'，代码:'000001'，日期:'2012-01-01'，价格:1000}
{market:'SZ'，code:'000001'，date:'2012-01-01'，价格:1000}
{market:'SH'，代码:'000001'，日期:'2012-01-02'，价格:1000}
{market:'SZ'，code:'000001'，date:'2012-01-02'，价格:1000}
{market:'SH'，代码:'000002'，日期:'2012-01-03'，价格:1000}
...

{market: 'SH', code: '000001', date: '2012-01-01', price: 1000}
{market: 'SZ', code: '000001', date: '2012-01-01', price: 1000}
{market: 'SH', code: '000001', date: '2012-01-02', price: 1000}
{market: 'SZ', code: '000001', date: '2012-01-02', price: 1000}
{market: 'SH', code: '000002', date: '2012-01-03',price: 1000}
...

此收藏集包含数千万个文档.

This collection contains tens of millions documents.

我想用两个键来调用distinct:

I want to call distinct with two keys:

collection.distinct('market', 'code');

并获得结果:

[{market:'SH'，code:'000001'}， {market:"SZ"，代码:"000001"}， {market:'SH'，代码:'000002'}]

[{market: 'SH', code:'000001'}, {market: 'SZ', code:'000001'}, {market: 'SH', code:'000002'}]

由于本机独特命令仅接受一个键，因此我尝试使用map-reduce来实现它.但是map-reduce太慢了，以至于无法原生识别.在我的一键式非重复测试中，map-reduce的花费是本机非重复的时间的十倍左右.
有没有一种有效的方法来实现多键区分开?

As native distinct command accept only one key, I try to implement it by using map-reduce. But map-reduce is far too slow to native distinct. In my one-key distinct test, map-reduce spend about ten times longer than native distinct.
Is there a efficient way to implement multikey distinct?

推荐答案

如果您愿意等待即将发布的MongoDB 2.2版本，则可以使用聚合框架高效地运行此查询:

If you are willing to wait for the upcoming 2.2 release of MongoDB, you can run this query efficiently using the aggregation framework:

collection = db.tb;
result = collection.aggregate( 
            [
                {"$group": { "_id": { market: "$market", code: "$code" } } }
            ]
        );
printjson(result);

在我的测试机上进行的上百万条记录的收集中，这耗时4秒，而map/reduce版本花费了超过一分钟的时间.

On a million-record collection on my test machine, this ran in 4 seconds, while the map/reduce version took over a minute.

这篇关于如何有效地执行“与众不同"?有多个键?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！