如何在Kryo中注册Receiver []? [英] How to register Receiver[] with Kryo?

问题描述

在将Spark 1.6应用程序移植到Spark 2.0.2的过程中，日志中存在以下投诉:

In the process of porting a Spark 1.6 app to Spark 2.0.2, there's this complaint in the log:

com.esotericsoftware.kryo.KryoException: java.lang.IllegalArgumentException: Class is not registered: org.apache.spark.streaming.receiver.Receiver[]
Note: To register this class use: kryo.register(org.apache.spark.streaming.receiver.Receiver[].class);

此操作失败，并出现，原因如下:java.lang.ClassNotFoundException:org/apache/spark/streaming/receiver/Receiver []/class :

sparkConf.set("spark.kryo.classesToRegister", "org.apache.spark.streaming.receiver.Receiver[].class")

此操作失败，并出现，原因如下:java.lang.ClassNotFoundException:org/apache/spark/streaming/receiver/Receiver [] :

sparkConf.set("spark.kryo.classesToRegister", "org.apache.spark.streaming.receiver.Receiver[]")

此操作失败，并且未注册类:org.apache.spark.streaming.receiver.Receiver [] :

sparkConf.set("spark.kryo.classesToRegister", "org.apache.spark.streaming.receiver.Receiver")

此操作失败，并且未注册类:org.apache.spark.streaming.receiver.Receiver [] :

sparkConf.registerKryoClasses(Array(
    classOf[org.apache.spark.streaming.receiver.Receiver[_]]
))

如何注册该课程?我已经可以在Kryo上注册其他课程，但不能注册这个课程.

How can I get this class registered? I've been able to register other classes with Kryo, but not this one.

在所有这些情况下，此设置均已完成:

In all these cases this setup is done:

sparkConf.set("spark.kryo.registrationRequired", "true")
sparkConf.set("spark.serializer", classOf[KryoSerializer].getName)
GraphXUtils.registerKryoClasses(sparkConf)

推荐答案

找到了一种适用于此的方法:

Found an approach that works here: Kryo serialization refuses to register class

简而言之，将 classOf [org.apache.spark.streaming.receiver.Receiver [_]] 更改为 classOf [Array [org.apache.spark.streaming.receiver].接收者[_]]]，

这篇关于如何在Kryo中注册Receiver []?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！