组合器和减速器可以不同吗? [英] combiner and reducer can be different?

问题描述

在许多 MapReduce 程序中，我看到一个 reducer 也被用作组合器.我知道这是因为这些程序的特殊性质.但我想知道它们是否可以不同.

In many MapReduce programs, I see a reducer being used as a combiner as well. I know this is because of the specific nature of those programs. But I am wondering if they can be different.

推荐答案

是的，combiner 可以不同于 Reducer，尽管您的 Combiner 仍将实现 Reducer 接口.组合器只能用于依赖于工作的特定情况.Combiner 将像 Reducer 一样运行，但仅作用于每个 Mapper 输出的 Key/Values 子集.

Yes, a combiner can be different to the Reducer, although your Combiner will still be implementing the Reducer interface. Combiners can only be used in specific cases which are going to be job dependent. The Combiner will operate like a Reducer, but only on the subset of the Key/Values output from each Mapper.

与 Reducer 不同，Combiner 将具有的一个约束是输入/输出键和值类型必须与 Mapper 的输出类型匹配.

One constraint that your Combiner will have, unlike a Reducer, is that the input/output key and value types must match the output types of your Mapper.

这篇关于组合器和减速器可以不同吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！