本地模式和集群中Spark程序的区别 [英] Spark program difference in local mode and cluster

问题描述

如果我编写一个spark程序并以独立模式运行它，并且要在群集中部署它，是否需要更改程序代码还是不需要更改代码?Spark编程是否与群集数量无关?

If i write a spark program and run it in stand alone mode and when I want to deploy it in a cluster, do I have to change my program codes or no change in codes needed? Is spark programming independent of number of clusters?

推荐答案

当您要运行时，只需将选项 master 设置为 yarn 或其他资源管理器它在群集上.如果要在本地运行它，只需使用 local [*] ，方法是使用与计算机核心相同的线程数.

You just need to set option master to yarn or other resource manager, when you want to run it on cluster. If you want to run it locally just use local[*] by using the number of threads which equal to your machine cores.

这篇关于本地模式和集群中Spark程序的区别的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！