hadoop:0 reducer 和 identity reducer 的区别? [英] hadoop: difference between 0 reducer and identity reducer?

问题描述

我只是想确认我对 0 reducer 和 identity reducer 之间区别的理解.

I am just trying to confirm my understanding of difference between 0 reducer and identity reducer.

• 0 reducer 表示将跳过 reduce 步骤，mapper 输出将是最终输出
• 身份缩减器意味着洗牌/排序仍然会发生吗?

推荐答案

你的理解是正确的.我将其定义如下:如果您不需要对地图结果进行排序 - 您设置 0 减少，并且该作业仅称为地图.
如果您需要对映射结果进行排序，但不需要任何聚合 - 您选择身份归约器.
为了完成这幅图，我们还有第三种情况:我们确实需要聚合，在这种情况下我们需要 reducer.

You understanding is correct. I would define it as following: If you do not need sorting of map results - you set 0 reduced,and the job is called map only.
If you need to sort the mapping results, but do not need any aggregation - you choose identity reducer.
And to complete the picture we have a third case : we do need aggregation and, in this case we need reducer.

这篇关于hadoop:0 reducer 和 identity reducer 的区别?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！