从App扩展到对象扩展之间的差异包含Scala中的main方法 [英] Diffrence between extends from App and object contain main method in scala

问题描述

我编写了示例spark-scala程序，用于从数据帧创建json元素列表.当我使用main方法执行时，它返回空列表，但是当我在没有扩展应用程序的对象的情况下执行时，它返回包含记录的列表. Scala对象中extends App和main方法有什么区别

I wrote sample spark-scala program for creating list of json elements from dataframe. when i executed with main method it returns empty list but when i executed without object that extends app it returns list that contains records. what is the difference between extends App and main method in scala object

object DfToMap {
def main(args: Array[String]): Unit = {
val spark: SparkSession = SparkSession.builder()
.appName("Rnd")
.master("local[*]")
.getOrCreate()
import spark.implicits._

val df = Seq(
(8, "bat"),
(64, "mouse"),
(27, "horse")
).toDF("number", "word")

val json = df.toJSON
val jsonArray = new util.ArrayList[String]()
json.foreach(f => jsonArray.add(f))
print(jsonArray)
}
}

它将返回空列表 但是下面的程序给了我带有记录的列表

It will return empty list But following program gives me list with records

object DfToMap extends App{
val spark: SparkSession = SparkSession.builder()
.appName("Rnd")
.master("local[*]")
.getOrCreate()
import spark.implicits._

val df = Seq(
(8, "bat"),
(64, "mouse"),
(27, "horse")
).toDF("number", "word")

val json = df.toJSON
val jsonArray = new util.ArrayList[String]()
json.foreach(f => jsonArray.add(f))
print(jsonArray)

}

推荐答案

TL; DR 这两个代码段都不是正确的Spark程序，但是一个比另一个更不正确.

TL;DR Both snippets are not correct Spark programs, but one is just more incorrect than the other.

您犯了两个错误，都在Spark入门资料中进行了解释.

You've made two mistakes, both explained in the introductory Spark materials.

• 由于其性质，Spark不支持扩展App的应用程序-快速入门-自包含的应用程序

请注意，应用程序应定义main()方法，而不是扩展scala.App. scala.App的子类可能无法正常工作.

Note that applications should define a main() method instead of extending scala.App. Subclasses of scala.App may not work correctly.

Spark不提供全局共享内存，因此不支持修改全局对象是否为闭包- 查看全文