什么是Map/Reduce? [英] What is Map/Reduce?

问题描述

我听到了很多有关map/reduce的知识，尤其是在Google的大规模并行计算系统的背景下.到底是什么?

I hear a lot about map/reduce, especially in the context of Google's massively parallel compute system. What exactly is it?

推荐答案

摘自Google MapReduce 研究出版物页面:

From the abstract of Google's MapReduce research publication page:

MapReduce是一种编程模型， 相关的实现 处理和生成大数据 套.用户指定地图功能 处理一个键/值对 产生一组中间体 键/值对和归约函数 合并所有中间值 与同一中间体相关 键.

MapReduce is a programming model and an associated implementation for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key.

MapReduce的优点是可以在多个处理节点(多个服务器)上并行执行处理，因此它是一个可以很好扩展的系统.

The advantage of MapReduce is that the processing can be performed in parallel on multiple processing nodes (multiple servers) so it is a system that can scale very well.

由于它基于功能编程模型，因此map和reduce每个步骤都没有任何副作用(map进程的每个子部分的状态和结果都不依赖于另一个)，因此可以将映射和精简的数据集分别在多个处理节点上分开.

Since it's based from the functional programming model, the map and reduce steps each do not have any side-effects (the state and results from each subsection of a map process does not depend on another), so the data set being mapped and reduced can each be separated over multiple processing nodes.

Joel的您的编程语言可以做到这一点吗?讨论了如何了解函数式编程对于Google提出MapReduce至关重要，MapReduce为其搜索引擎提供了强大动力.如果您不熟悉函数式编程及其如何支持可扩展代码，那么这是一本很好的读物.

Joel's Can Your Programming Language Do This? piece discusses how understanding functional programming was essential in Google to come up with MapReduce, which powers its search engine. It's a very good read if you're unfamiliar with functional programming and how it allows scalable code.

另请参阅:维基百科:MapReduce

相关问题:请简单地解释mapreduce

这篇关于什么是Map/Reduce?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！