Spark 2.x Scala 2.1.1自定义编码器类类型不匹配 [英] Spark 2.x scala 2.1.1 custom encoder class type mismatch
本文介绍了Spark 2.x Scala 2.1.1自定义编码器类类型不匹配的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我正在使用此代码在scala 2.1.1中为spark 2.x定义编码器:
I am using this code to define an encoder for spark 2.x, in scala 2.1.1:
import org.apache.spark.sql.Encoder
import org.apache.spark.sql.types.StructType
class LogProcessorMessageEncoder extends Encoder[LogProcessorMessage] {
override def schema: StructType = null
override def clsTag: Class[LogProcessorMessage] = classOf[LogProcessorMessage]
}
object LogProcessorMessage {}
class LogProcessorMessage extends Serializable {}
并且IntelliJ说很好,但是对clsTag方法的重写会导致编译器错误:
And IntelliJ says it is fine, but the override of the clsTag method results in a compiler error:
Error:(20, 15) overriding method clsTag in trait Encoder of type => scala.reflect.ClassTag[dvn.aast.telemetry.scala.stream.LogProcessorMessage];
method clsTag has incompatible type
override def clsTag: Class[LogProcessorMessage] = classOf[LogProcessorMessage]
推荐答案
clsTag
必须是 ClassTag
而不是 Class
:
class LogProcessorMessageEncoder extends Encoder[LogProcessorMessage] {
override def clsTag: ClassTag[LogProcessorMessage] = ClassTag(classOf[LogProcessorMessage])
override def schema: StructType = null
}
这篇关于Spark 2.x Scala 2.1.1自定义编码器类类型不匹配的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文