在 Scala 中将数据帧作为可选函数参数传递 [英] Passing data frame as optional function parameter in Scala
问题描述
有没有办法在 Scala 中将数据框作为可选的输入函数参数传递?例如:
Is there a way that I can pass a data frame as an optional input function parameter in Scala? Ex:
def test(sampleDF: DataFrame = df.sqlContext.emptyDataFrame): DataFrame = {
}
df.test(sampleDF)
虽然我在这里传递了一个有效的数据框,但它总是被分配给一个空的数据框,我该如何避免这种情况?
Though I am passing a valid data frame here , it is always assigned to an empty data frame, how can I avoid this?
推荐答案
是的,您可以将 dataframe
作为参数传递给函数
Yes you can pass dataframe
as a parameter to a function
假设你有一个 dataframe
作为
lets say you have a dataframe
as
import sqlContext.implicits._
val df = Seq(
(1, 2, 3),
(1, 2, 3)
).toDF("col1", "col2", "col3")
这是
+----+----+----+
|col1|col2|col3|
+----+----+----+
|1 |2 |3 |
|1 |2 |3 |
+----+----+----+
您可以将其传递给如下函数
you can pass it to a function as below
import org.apache.spark.sql.DataFrame
def test(sampleDF: DataFrame): DataFrame = {
sampleDF.select("col1", "col2") //doing some operation in dataframe
}
val testdf = test(df)
testdf
将是
+----+----+
|col1|col2|
+----+----+
|1 |2 |
|1 |2 |
+----+----+
已编辑
正如 Eliasah 指出的,@Garipaso 想要可选参数.这可以通过将函数定义为
As eliasah pointed out that @Garipaso wanted optional argument. This can be done by defining the function as
def test(sampleDF: DataFrame = sqlContext.emptyDataFrame): DataFrame = {
if(sampleDF.count() > 0) sampleDF.select("col1", "col2") //doing some operation in dataframe
else sqlContext.emptyDataFrame
}
如果我们传递一个有效的数据帧
If we pass a valid dataframe as
test(df).show(false)
它将输出为
+----+----+
|col1|col2|
+----+----+
|1 |2 |
|1 |2 |
+----+----+
但是如果我们不传递参数为
But if we don't pass argument as
test().show(false)
我们会得到空数据框
++
||
++
++
希望回答对你有帮助
这篇关于在 Scala 中将数据帧作为可选函数参数传递的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!