使用MapReduce查找数字的平均值 [英] Find the average of numbers using MapReduce

问题描述

我一直在尝试编写一些代码来使用MapReduce查找数字的平均值。

我试图使用全局计数器来达到我的目标，但我不是能够在我的Mapper的 map 方法中设置计数器值，我也无法检索 reduce


我是否必须在 map 中使用全局计数器（例如，通过使用提供的 Reporter ）的 incrCounter（key，amount）或者你会建议任何不同的逻辑来获得一些数字的平均值？


解决方案

逻辑非常简单：
如果所有数字都具有相同的键，那么映射器将使用该键发送所有想要查找其平均值的值。因此，在reducer中，您可以对迭代器中的值进行求和。然后，您可以在迭代器工作的编号时间保留一个计数器，从而解决了需要对多少个项目进行平均的问题。最后，在迭代器之后，您可以通过将总和除以项目数量来找到平均值。

请小心，此逻辑不起作用如果combiner类被设置为与reducer相同的类...

I have been trying to write some code to find the average of numbers using MapReduce.

I am trying to use global counters to reach my goal but I am not able to set the counter value in the map method of my Mapper and I am also not able to retrive the counter value in the reduce method of my Reducer.

Do I have to use a global counter in map anyway (e.g. by using incrCounter(key, amount) of the provided Reporter)? Or would you suggest any different logic to get the average of some numbers?

解决方案

The logic is quite simple: If all the number have the same key, then the mapper sent all the values you want to find the average of with that same key. Because of this, in the reducer you can sum the values in the iterator. You can then keep a counter on number time the iterator works, which solves the issue of how many items are to be averaged. Finally, after the iterator, you can find the average by dividing the sum by the number of items.

Be careful, this logic will not work if the combiner class is set as the same class as reducer...

这篇关于使用MapReduce查找数字的平均值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！