setMaster`local [*]`在spark中是什么意思? [英] What does setMaster `local[*]` mean in spark?

问题描述

我发现了一些代码可以在本地启动spark

I found some code to start spark locally with:

val conf = new SparkConf().setAppName("test").setMaster("local[*]")
val ctx = new SparkContext(conf)

[*]是什么意思?

推荐答案

来自 doc :

./bin/spark-shell --master local[2]

--master选项指定分布式的主URL 集群，或者local可以在一个线程中本地运行，或者local[N]可以运行 在本地具有N个线程.您应该首先使用本地进行测试.

The --master option specifies the master URL for a distributed cluster, or local to run locally with one thread, or local[N] to run locally with N threads. You should start by using local for testing.

并从此处:

local[*]在本地运行Spark，其工作线程数与逻辑线程数相同 机器上的核心.

local[*] Run Spark locally with as many worker threads as logical cores on your machine.

这篇关于setMaster`local [*]`在spark中是什么意思?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！