组合器和减速器可以不同？ [英] combiner and reducer can be different?

问题描述

在很多MapReduce程序中，我看到一个reducer也被用作组合器。我知道这是因为这些计划的具体性质。但我想知道他们是否会有所不同。

解决方案

是的，组合器可以与Reducer不同，但Combiner仍然会实现Reducer接口。合并器只能用于需要依赖工作的特定情况。组合器将像减速器一样运行，但只能在每个映射器的键/值输出的子集上运行。

Combiner具有的一个约束条件与Reducer不同，是输入/输出键和值类型必须匹配Mapper的输出类型。

In many MapReduce programs, I see a reducer being used as a combiner as well. I know this is because of the specific nature of those programs. But I am wondering if they can be different.

解决方案

Yes, a combiner can be different to the Reducer, although your Combiner will still be implementing the Reducer interface. Combiners can only be used in specific cases which are going to be job dependent. The Combiner will operate like a Reducer, but only on the subset of the Key/Values output from each Mapper.

One constraint that your Combiner will have, unlike a Reducer, is that the input/output key and value types must match the output types of your Mapper.

这篇关于组合器和减速器可以不同？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！