MapReduce结果似乎限制为100? [英] MapReduce results seem limited to 100?

问题描述

我在MongoDB和python中玩Map Reduce，遇到了一个奇怪的限制.我只是想计算书"记录的数量.当记录少于100条时，它会起作用，但是当记录超过100条时，由于某种原因，计数会重置.

I'm playing around with Map Reduce in MongoDB and python and I've run into a strange limitation. I'm just trying to count the number of "book" records. It works when there are less than 100 records but when it goes over 100 records the count resets for some reason.

这是我的MR代码和一些示例输出:

Here is my MR code and some sample outputs:

var M = function () {
book = this.book;
emit(book, {count : 1});
}

var R = function (key, values) {
var sum = 0;
values.forEach(function(x) {
sum += 1;
});
var result = {
count : sum 
};
return result;
}

记录数为99时的MR输出:

MR output when record count is 99:

{u'_id': u'superiors', u'value': {u'count': 99}}

记录数为101时的MR输出:

MR output when record count is 101:

{u'_id': u'superiors', u'value': {u'count': 2.0}}

有什么想法吗?

推荐答案

您的reduce函数应该对count值求和，而不仅仅是为每个值添加1.否则，reduce的输出不能正确地用作另一个reduce的输入.尝试以下方法:

Your reduce function should be summing up the count values, not just adding 1 for each value. Otherwise the output of a reduce can't properly be used as input back into another reduce. Try this instead:

var R = function (key, values) {
  var sum = 0;
  values.forEach(function(x) {
    sum += x.count;
  });
  var result = {
    count : sum 
  };
  return result;
}

这篇关于MapReduce结果似乎限制为100?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！