相同类型的apache的火花类型不匹配（字符串） [英] Apache Spark type mismatch of the same type (String)

问题描述

编辑：答：这是创建一个冲突的JAR文件
相关的职位是：<一href=\"http://stackoverflow.com/questions/29375027/must-include-log4j-but-it-is-causing-errors-in-apache-spark-shell-how-to-avoid\">Must包括log4j的，但它造成的Apache星火外壳的错误。如何避免错误？

执行以下操作：

val numOfProcessors:Int = 2
val filePath:java.lang.String = "s3n://somefile.csv"
var rdd:org.apache.spark.rdd.RDD[java.lang.String] = sc.textFile(filePath, numOfProcessors)

我得到

    error: type mismatch;
 found   : org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.RDD[String]
 required: org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.RDD[String]
       var rdd:org.apache.spark.rdd.RDD[java.lang.String] = sc.textFile(filePath, numOfProcessors)

---

编辑：第二种情况

---

Second case

val numOfProcessors = 2
val filePath = "s3n://somefile.csv"
var rdd = sc.textFile(filePath, numOfProcessors) //OK!

def doStuff(rdd: RDD[String]): RDD[String] = {rdd}

doStuff(rdd)

我得到：

error: type mismatch;
 found   : org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.RDD[String]
 required: org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.org.apache.spark.rdd.RDD[String]
              doStuff(rdd)
                      ^

没有评论...

任何想法，为什么我得到这个错误？

Any ideas why I get this error ?

推荐答案

的问题是，创建一个冲突的JAR文件。

The problem was a JAR file that created a conflict.

这篇关于相同类型的apache的火花类型不匹配（字符串）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！