hadoop:0 reducer 和 identity reducer 的区别? [英] hadoop: difference between 0 reducer and identity reducer?
本文介绍了hadoop:0 reducer 和 identity reducer 的区别?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我只是想确认我对 0 reducer 和 identity reducer 之间区别的理解.
I am just trying to confirm my understanding of difference between 0 reducer and identity reducer.
- 0 reducer 表示将跳过 reduce 步骤,mapper 输出将是最终输出
- 身份缩减器意味着洗牌/排序仍然会发生吗?
推荐答案
你的理解是正确的.我将其定义如下:如果您不需要对地图结果进行排序 - 您设置 0 减少,并且该作业仅称为地图.
如果您需要对映射结果进行排序,但不需要任何聚合 - 您选择身份归约器.
为了完成这幅图,我们还有第三种情况:我们确实需要聚合,在这种情况下我们需要 reducer.
You understanding is correct. I would define it as following:
If you do not need sorting of map results - you set 0 reduced,and the job is called map only.
If you need to sort the mapping results, but do not need any aggregation - you choose identity reducer.
And to complete the picture we have a third case : we do need aggregation and, in this case we need reducer.
这篇关于hadoop:0 reducer 和 identity reducer 的区别?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文