如何将 Column.isin 与列表一起使用? [英] How to use Column.isin with list?
本文介绍了如何将 Column.isin 与列表一起使用?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
val items = List("a", "b", "c")
sqlContext.sql("select c1 from table")
.filter($"c1".isin(items))
.collect
.foreach(println)
上面的代码抛出以下异常.
The code above throws the following exception.
Exception in thread "main" java.lang.RuntimeException: Unsupported literal type class scala.collection.immutable.$colon$colon List(a, b, c)
at org.apache.spark.sql.catalyst.expressions.Literal$.apply(literals.scala:49)
at org.apache.spark.sql.functions$.lit(functions.scala:89)
at org.apache.spark.sql.Column$$anonfun$isin$1.apply(Column.scala:642)
at org.apache.spark.sql.Column$$anonfun$isin$1.apply(Column.scala:642)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245)
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:35)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:245)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at org.apache.spark.sql.Column.isin(Column.scala:642)
下面是我修复它的尝试.它编译并运行但不返回任何匹配项.不知道为什么.
Below is my attempt to fix it. It compiles and runs but doesn't return any match. Not sure why.
val items = List("a", "b", "c").mkString("\"","\",\"","\"")
sqlContext.sql("select c1 from table")
.filter($"c1".isin(items))
.collect
.foreach(println)
推荐答案
根据文档,isin
需要一个可变参数,而不是一个列表.List在这里实际上是一个令人困惑的名字.您可以尝试将您的 List 转换为 vararg ,如下所示:
According to documentation, isin
takes a vararg, not a list. List is actually a confusing name here. You can try converting your List to vararg like this:
val items = List("a", "b", "c")
sqlContext.sql("select c1 from table")
.filter($"c1".isin(items:_*))
.collect
.foreach(println)
您使用 mkString 的变体可以编译,因为单个 String 也是可变参数(参数数量等于 1),但这可能不是您想要实现的.
Your variant with mkString compiles, because one single String is also a vararg (with number of arguments equal to 1), but it is proably not what you want to achieve.
这篇关于如何将 Column.isin 与列表一起使用?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文