MapReduce shuffle/sort 方法 [英] MapReduce shuffle/sort method

问题描述

有点奇怪的问题，但有谁知道 MapReduce 在 shuffle/sort 的排序部分使用什么样的排序?我会考虑合并或插入(与整个 MapReduce 范例保持一致)，但我不确定.

Somewhat of an odd question, but does anyone know what kind of sort MapReduce uses in the sort portion of shuffle/sort? I would think merge or insertion (in keeping with the whole MapReduce paradigm), but I'm not sure.

推荐答案

这是快速排序，之后排序的中间输出合并在一起.快速排序检查递归深度并在它太深时放弃.如果是这种情况，则使用堆排序.

It's Quicksort, afterwards the sorted intermediate outputs get merged together. Quicksort checks the recursion depth and gives up when it is too deep. If this is the case, Heapsort is used.

看看 Quicksort 类:

Have a look at the Quicksort class:

org.apache.hadoop.util.QuickSort

您可以通过 hadoop-default.xml.

这篇关于MapReduce shuffle/sort 方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！