MPI 比 MapReduce 更适合哪些场景? [英] What are some scenarios for which MPI is a better fit than MapReduce?

问题描述

据我所知，MPI 使我能够更好地控制集群中不同节点的通信方式.

As far as I understand, MPI gives me much more control over how exactly different nodes in the cluster will communicate.

在 MapReduce/Hadoop 中，每个节点进行一些计算，与其他节点交换数据，然后整理其结果的分区.看起来很简单，但由于您可以迭代该过程，因此即使是 K-means 或 PageRank 等算法也能很好地拟合模型.在具有调度局部性的分布式文件系统上，性能显然是不错的.相比之下，MPI 让我可以明确控制节点如何相互发送消息.

In MapReduce/Hadoop, each node does some computation, exchanges data with other nodes, and then collates its partition of results. Seems simple, but since you can iterate the process, even algorithms like K-means or PageRank fit the model quite well. On a distributed file system with locality of scheduling, the performance is apparently good. In comparison, MPI gives me explicit control over how nodes send messages to each other.

谁能描述一个集群编程场景，其中更通用的 MPI 模型明显优于更简单的 MapReduce 模型?

Can anyone describe a cluster programming scenario where the more general MPI model is an obvious advantage over the simpler MapReduce model?

推荐答案

几乎所有的科学代码——有限差分、有限元等.哪种导致循环答案，任何分布式程序都不容易映射使用更通用的 MPI 模型可以更好地实现 MapReduce.不确定这对你有多大帮助，我会在发布后立即拒绝这个答案.

Almost any scientific code -- finite differences, finite elements, etc. Which kind of leads to the circular answer, that any distributed program which doesn't easily map to MapReduce would be better implemented with a more general MPI model. Not sure that's much help to you, I'll downvote this answer right after I post it.

这篇关于MPI 比 MapReduce 更适合哪些场景?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！