解决方法斯卡拉RDD不是协变 [英] Workaround for Scala RDD not being covariant

问题描述

我试图写一个函数来对RDD [SEQ [字符串]对象，例如操作：

I'm trying to write a function to operate on RDD[Seq[String]] objects, e.g.:

def foo(rdd: RDD[Seq[String]]) = { println("hi") }

此功能不能呼吁类型RDD对象[数组[字符串]]：

This function cannot be called on objects of type RDD[Array[String]]:

val testRdd : RDD[Array[String]] = sc.textFile("somefile").map(_.split("\\|", -1))
foo(testRdd)

->
error: type mismatch;
found   : org.apache.spark.rdd.RDD[Array[String]]
required: org.apache.spark.rdd.RDD[Seq[String]]

我想这是因为RDD不是协变的。

I guess that's because RDD isn't covariant.

我已经尝试了一堆富的定义来解决这个问题。其中只有一个人编译：

I've tried a bunch of definitions of foo to get around this. Only one of them has compiled:

def foo2[T[String] <: Seq[String]](rdd: RDD[T[String]]) = { println("hi") }

但它仍然是破：

foo2(testRdd)


->
<console>:101: error: inferred type arguments [Array] do not conform to method foo2's type
parameter bounds [T[String] <: Seq[String]]
          foo2(testRdd)
          ^
<console>:101: error: type mismatch;
found   : org.apache.spark.rdd.RDD[Array[String]]
required: org.apache.spark.rdd.RDD[T[String]]

任何想法，我怎么能解决此问题？这一切发生在星火外壳。

Any idea how I can work around this? This is all taking place in the Spark shell.

推荐答案

有关这个，你可以使用绑定一个视图

For this you can use a view bound.

阵列不是 SEQ ，但它可以的查看的一个 SEQ 。

Array is not a Seq, but it can be viewed as a Seq.

def foo[T <% Seq[String]](rdd: RDD[T]) = ???

的＆LT;％表示， T 可以作为序列[字符串进行查看] 这样，每当您使用 T 则序列[字符串] 方法 T 将被转换为序列[字符串] 。

The <% says that T can be viewed as a Seq[String] so that whenever you use a Seq[String] method on T then T will be converted to Seq[String].

有关数组[A] 被视为序列[A] 需要有在隐函数范围可阵列取值转换为 SEQ 秒。由于约努茨G.斯坦说，它存在于阶。preDEF

For Array[A] to be viewed as Seq[A] there needs to be an implicit function in scope that can convert Arrays to Seqs. As Ionuț G. Stan said, it exists in scala.Predef.

这篇关于解决方法斯卡拉RDD不是协变的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！