Apache Spark案例在不同的列上具有多个when子句 [英] Apache spark case with multiple when clauses on different columns
本文介绍了Apache Spark案例在不同的列上具有多个when子句的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
给出以下结构:
val df = Seq("Color", "Shape", "Range","Size").map(Tuple1.apply).toDF("color")
val df1 = df.withColumn("Success", when($"color"<=> "white", "Diamond").otherwise(0))
我想在上面再写一个WHEN条件,其中size> 10和Shape列 值是菱形,然后将钻石"值插入到其他列0中.我尝试如下操作,但失败了
I want to write one more WHEN condition at above where size > 10 and Shape column value is Rhombus then "Diamond" value should be inserted to the column else 0. I tried like below but it's failing
val df1 = df.withColumn("Success", when($"color" <=> "white", "Diamond").otherwise(0)).when($"size">10)
请只建议我在scala中使用dataframe选项.带有sqlContext的Spark-SQL对我没有帮助.
Please suggest me with only dataframe option with scala. Spark-SQL with sqlContext is not helpful idea for me.
谢谢!
推荐答案
You can chain the when
similar to the example in https://spark.apache.org/docs/latest/api/java/org/apache/spark/sql/Column.html#when-org.apache.spark.sql.Column-java.lang.Object-
available since (1.4.0)
// Scala:
people.select(when(people("gender") === "male", 0)
.when(people("gender") === "female", 1)
.otherwise(2))
您的示例:
val df1 = df.withColumn("Success",
when($"color" <=> "white", "Diamond")
.when($"size" > 10 && $"shape" === "Rhombus", "Diamond")
.otherwise(0))
这篇关于Apache Spark案例在不同的列上具有多个when子句的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文