“AssertionError: assertion failed: unsafe symbol Unstable"是什么意思? [英] What does this mean "AssertionError: assertion failed: unsafe symbol Unstable"?

问题描述

我在 Java 8 中使用 spark-sql-2.4.1 版本.

I am using spark-sql-2.4.1 version with Java 8.

我有 raw_df 列eventTs"，这是 Long 数据类型的时间戳.我正在尝试将其转换回时间戳，如下所示:

I have raw_df with column "eventTs" which is timestamp in Long data type. I am trying to convert it back to Timestamp as below :

Dataset<Row> modified_df = raw_df
  .withColumn("eventTimeStamp", ( col("eventTs").divide(1000) ).cast(DataTypes.TimestampType))

当使用 ( col("eventTs").divide(1000) ).cast(DataTypes.TimestampType)或 ( col("eventTs").divide(lit(1000)) ).cast(DataTypes.TimestampType)

它工作正常，但低于 AssertionError

It is working fine but it is throwing below AssertionError

java.lang.AssertionError: assertion failed: unsafe symbol Unstable (child of package InterfaceStability) in runtime reflection universe
    at scala.reflect.internal.Symbols$Symbol.<init>(Symbols.scala:184)
    at scala.reflect.internal.Symbols$TypeSymbol.<init>(Symbols.scala:3009)
    at scala.reflect.internal.Symbols$ClassSymbol.<init>(Symbols.scala:3201)
    at scala.reflect.internal.Symbols$StubClassSymbol.<init>(Symbols.scala:3496)
    at scala.reflect.internal.Symbols$Symbol.newStubSymbol(Symbols.scala:498)
    at scala.reflect.internal.pickling.UnPickler$Scan.readExtSymbol$1(UnPickler.scala:258)
    at scala.reflect.internal.pickling.UnPickler$Scan.readSymbol(UnPickler.scala:284)
    at scala.reflect.internal.pickling.UnPickler$Scan.readSymbolRef(UnPickler.scala:649)
    at scala.reflect.internal.pickling.UnPickler$Scan.readType(UnPickler.scala:417)
    at scala.reflect.internal.pickling.UnPickler$Scan$$anonfun$readTypeRef$1.apply(UnPickler.scala:658)
    at scala.reflect.internal.pickling.UnPickler$Scan$$anonfun$readTypeRef$1.apply(UnPickler.scala:658)
    at scala.reflect.internal.pickling.UnPickler$Scan.at(UnPickler.scala:179)
    at scala.reflect.internal.pickling.UnPickler$Scan.readTypeRef(UnPickler.scala:658)
    at scala.reflect.internal.pickling.UnPickler$Scan.readAnnotationInfo(UnPickler.scala:492)
    at scala.reflect.internal.pickling.UnPickler$Scan.readSymbolAnnotation(UnPickler.scala:515)
    at scala.reflect.internal.pickling.UnPickler$Scan.run(UnPickler.scala:97)
    at scala.reflect.internal.pickling.UnPickler.unpickle(UnPickler.scala:38)
    at scala.reflect.runtime.JavaMirrors$JavaMirror.unpickleClass(JavaMirrors.scala:619)
    at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply$mcV$sp(SymbolLoaders.scala:28)
    at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply(SymbolLoaders.scala:25)
    at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply(SymbolLoaders.scala:25)
    at scala.reflect.internal.SymbolTable.slowButSafeEnteringPhaseNotLaterThan(SymbolTable.scala:263)
    at scala.reflect.runtime.SymbolLoaders$TopClassCompleter.complete(SymbolLoaders.scala:25)
    at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$10.scala$reflect$runtime$SynchronizedSymbols$SynchronizedSymbol$$super$info(SynchronizedSymbols.scala:189)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
    at scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
    at scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$10.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:189)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.info(SynchronizedSymbols.scala:127)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$10.info(SynchronizedSymbols.scala:189)
    at scala.reflect.internal.SymbolTable.openPackageModule(SymbolTable.scala:286)
    at scala.reflect.internal.SymbolTable.openPackageModule(SymbolTable.scala:341)
    at scala.reflect.runtime.SymbolLoaders$LazyPackageType$$anonfun$complete$2.apply$mcV$sp(SymbolLoaders.scala:74)
    at scala.reflect.runtime.SymbolLoaders$LazyPackageType$$anonfun$complete$2.apply(SymbolLoaders.scala:71)
    at scala.reflect.runtime.SymbolLoaders$LazyPackageType$$anonfun$complete$2.apply(SymbolLoaders.scala:71)
    at scala.reflect.internal.SymbolTable.slowButSafeEnteringPhaseNotLaterThan(SymbolTable.scala:263)
    at scala.reflect.runtime.SymbolLoaders$LazyPackageType.complete(SymbolLoaders.scala:71)
    at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1514)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$1.scala$reflect$runtime$SynchronizedSymbols$SynchronizedSymbol$$super$info(SynchronizedSymbols.scala:174)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$info$1.apply(SynchronizedSymbols.scala:127)
    at scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
    at scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$1.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:174)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.info(SynchronizedSymbols.scala:127)
    at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$1.info(SynchronizedSymbols.scala:174)
    at scala.reflect.internal.Types$TypeRef.thisInfo(Types.scala:2194)
    at scala.reflect.internal.Types$TypeRef.baseClasses(Types.scala:2199)
    at scala.reflect.internal.tpe.FindMembers$FindMemberBase.<init>(FindMembers.scala:17)
    at scala.reflect.internal.tpe.FindMembers$FindMember.<init>(FindMembers.scala:219)
    at scala.reflect.internal.Types$Type.scala$reflect$internal$Types$Type$$findMemberInternal$1(Types.scala:1014)
    at scala.reflect.internal.Types$Type.findMember(Types.scala:1016)
    at scala.reflect.internal.Types$Type.memberBasedOnName(Types.scala:631)
    at scala.reflect.internal.Types$Type.member(Types.scala:600)
    at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:48)
    at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:45)
    at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:66)
    at scala.reflect.internal.Mirrors$RootsBase.staticModuleOrClass(Mirrors.scala:77)
    at scala.reflect.internal.Mirrors$RootsBase.staticModule(Mirrors.scala:161)
    at scala.reflect.internal.Mirrors$RootsBase.staticModule(Mirrors.scala:22)
    at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$schemaFor$1$$typecreator48$1.apply(ScalaReflection.scala:735)
    at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe$lzycompute(TypeTags.scala:232)
    at scala.reflect.api.TypeTags$WeakTypeTagImpl.tpe(TypeTags.scala:232)
    at org.apache.spark.sql.catalyst.ScalaReflection$class.localTypeOf(ScalaReflection.scala:921)
    at org.apache.spark.sql.catalyst.ScalaReflection$.localTypeOf(ScalaReflection.scala:46)
    at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$schemaFor$1.apply(ScalaReflection.scala:735)
    at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$schemaFor$1.apply(ScalaReflection.scala:724)
    at scala.reflect.internal.tpe.TypeConstraints$UndoLog.undo(TypeConstraints.scala:56)
    at org.apache.spark.sql.catalyst.ScalaReflection$class.cleanUpReflectionObjects(ScalaReflection.scala:906)
    at org.apache.spark.sql.catalyst.ScalaReflection$.cleanUpReflectionObjects(ScalaReflection.scala:46)
    at org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:723)
    at org.apache.spark.sql.catalyst.ScalaReflection$.schemaFor(ScalaReflection.scala:720)
    at org.apache.spark.sql.catalyst.expressions.Literal$$anonfun$create$1.apply(literals.scala:160)
    at org.apache.spark.sql.catalyst.expressions.Literal$$anonfun$create$1.apply(literals.scala:159)
    at scala.util.Try$.apply(Try.scala:192)
    at org.apache.spark.sql.catalyst.expressions.Literal$.create(literals.scala:159)
    at org.apache.spark.sql.functions$.typedLit(functions.scala:127)
    at org.apache.spark.sql.functions$.lit(functions.scala:110)
    at org.apache.spark.sql.Column.$div(Column.scala:747)
    at org.apache.spark.sql.Column.divide(Column.scala:762)
    at 

这里有什么问题?如何解决这个问题?

What is wrong here ? how to fix this?

推荐答案

也许您的类路径中缺少 spark-tags_2.11.jar?这个 JAR 定义了 InterfaceStability 包.

Maybe you're missing spark-tags_2.11.jar from your classpath? This JAR defines InterfaceStability package.

这篇关于“AssertionError: assertion failed: unsafe symbol Unstable"是什么意思?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！