蜂巢崩溃与java.lang.IncompatibleClassChangeError [英] Hive crashing with java.lang.IncompatibleClassChangeError
问题描述
使用
java.lang.IncompatibleClassChangeError: Class com.google.common.collect.ImmutableSortedMap does not implement the requested interface java.util.NavigableMap
show tables之类的命令都可以正常运行，并且还可以从CLI正常加载数据.
Commands like show tables all run fine and data is loaded ok from the CLI as well.
检查了其他各种命令，例如数据已加载等.使用MySQL作为MySQL-connector-java-5.1.47.jar的元存储.唯一的观察结果是有时候我会得到
Checked various other commands and e.g. data is loaded etc. Uses MySQL as metastore with MySQL-connector-java-5.1.47.jar. The only other observation is that sometimes I get
WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored
其他人似乎也能得到，并且似乎不会在这里影响我. 有人也看到了吗?非常感谢帮助...
which other people seem to get as well and seems not to impact me here. Anybody seen this as well? Help greatly appreciated ...
2019-04-02 16:24:41,643 INFO metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore
2019-04-02 16:24:41,645 INFO HiveMetaStore.audit: ugi=fdai0145 ip=unknown-ip-addr cmd=Done cleaning up thread local RawStore
Exception in thread "main" java.lang.IncompatibleClassChangeError: Class com.google.common.collect.ImmutableSortedMap does not implement the requested interface java.util.NavigableMap
at org.apache.calcite.schema.Schemas.gatherLattices(Schemas.java:498)
at org.apache.calcite.schema.Schemas.getLatticeEntries(Schemas.java:492)
at org.apache.calcite.jdbc.CalciteConnectionImpl.init(CalciteConnectionImpl.java:153)
at org.apache.calcite.jdbc.Driver$1.onConnectionInit(Driver.java:109)
at org.apache.calcite.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:139)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:150)
at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:111)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1414)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1430)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:450)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12161)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:330)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:285)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:659)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1826)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1773)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1768)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:214)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:402)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236).
推荐答案
也许这是一个较晚的答案，但我确实遇到了同样的问题.就我而言，我发现hive-exec Maven工件的jar文件正在遮盖Google集合框架.现在，由于我已经看到其他Hadoop/Hive工件也使用了Google Guava(如果我没记错的话，是版本11)，所以方解石很有可能会找到ImmutableSortedMap的错误类定义(来自Guava 11) ).
Perhaps it's a late answer, but I did run into the same issue. In my case, I found that the hive-exec Maven artifact's jar file is shading the Google collections framework. Now, since I've seen that other Hadoop/Hive artifacts also make use of Google Guava (version 11 if I'm not mistaken), there's a good chance that calcite will find the wrong class definition for ImmutableSortedMap (from Guava 11).
对我来说，将番石榴从我的代码使用的Hadoop/Hive工件中排除后，方解石就可以从Google集合中找到正确的类版本.
For me, excluding guava from the Hadoop/Hive artifacts that my code uses made calcite find the correct class version from Google collections.
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-minicluster</artifactId>
<version>${hadoop.version}</version>
<exclusions>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-exec</artifactId>
<version>${hive.version}</version>
<exclusions>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-jdbc</artifactId>
<version>${hive.version}</version>
<exclusions>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
</exclusions>
</dependency>
这可能是应该向Hive项目报告的问题，因为这类类路径冲突错误很难诊断.内部阴影的工件应具有项目自己的程序包前缀，以指示所涉及的外部代码的显式阴影.
This is probably an issue that should be reported to the Hive project, since these kinds of class path collision errors are hard to diagnose. Internally shaded artifacts should have the project's own package prefix to indicate explicit shading of the external code in question.
哦，很好.希望这会有所帮助.
Oh well. Hope this helps.
这篇关于蜂巢崩溃与java.lang.IncompatibleClassChangeError的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
