MapReduce 替代方案 [英] MapReduce alternatives

查看:66
本文介绍了MapReduce 替代方案的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

是否有 MapReduce(Google、Hadoop)的替代范例?有没有其他合理的方式来拆分 &合并大问题?

Are there any alternative paradigms to MapReduce (Google, Hadoop)? Is there any other reasonable way how to split & merge big problems?

推荐答案

肯定的.例如,查看 批量同步并行.Map/Reduce 实际上是一种非常有限的减少问题的方法,但是这种限制使它可以在 Hadoop 等框架中进行管理.问题是,将您的问题压入 Map/Reduce 设置是否更容易,或者是否更容易创建特定于域的并行化方案并且必须自己处理所有实现细节.事实上,Pig 只是 Hadoop 之上的一个抽象层,它可以自动将许多标准问题从非 Map-Reduce-y 转换为 Map-Reduce-compatible.

Definitively. Check out, for example, Bulk Synchronous Parallel. Map/Reduce is in fact a very restricted way of reducing problems, however that restriction makes it manageable in a framework like Hadoop. The question is if it is less trouble to press your problem into a Map/Reduce setting, or if its easier to create a domain-specific parallelization scheme and having to take care of all the implementation details yourself. Pig, in fact, is only an abstraction layer on top of Hadoop which automates many standard problem transformations from not-Map-Reduce-y to Map-Reduce-compatible.

编辑 26.1.13:找到一个 不错的最新版本此处概览

Edit 26.1.13: Found a nice up-to-date overview here

这篇关于MapReduce 替代方案的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆