Zeppelin 0.7.2版本不支持spark 2.2.0 [英] Zeppelin 0.7.2 version does not support spark 2.2.0

问题描述

如何降级spark版本？什么可能是其他解决方案？我必须使用spark会话连接我的配置单元表来激发。但是zeppelin不支持spark版本。

解决方案

这里有两个原因。

<1> Zeppelin 0.7.2将spark 2.2+标记为不支持的版本。

因为只有火花解释器（2.2版本）需要java 8，Zeppelin不需要它。

总之

1. 修改这里，如果你想在Zeppelin分支上打出2.2的话就重新编译-0.7


2. 并且使用JDK 8作为spark解释器。

How to downgrade the spark version? What could be the other solutions? I have to connect my hive tables to spark using spark session. But the spark version is not supported by zeppelin.

解决方案

Here are 2 reasons.

[1] Zeppelin 0.7.2 marked spark 2.2+ as the unsupported version.

https://github.com/apache/zeppelin/blob/v0.7.2/spark/src/main/java/org/apache/zeppelin/spark/SparkVersion.java#L40

public static final SparkVersion UNSUPPORTED_FUTURE_VERSION = SPARK_2_2_0;

[2] Even if you change the const and build again, you might fail to run Zeppelin 0.7.2 with Spark 2.2

• https://spark.apache.org/releases/spark-release-2-2-0.html

Spark dropped support for Java 7 and Zeppelin 0.7.2 was built with JDK 7. So you need to rebuild it with JDK 8.

One work around you can use is, specifying JAVA_HOME in the spark interpreter (for 2.2) as commented here

• https://github.com/apache/zeppelin/pull/2486#issuecomment-314954959

That works because only spark interpreter (for 2.2) requires java 8 and Zeppelin doesn't need it.

In short

1. Modify here and rebuild if you want to spark 2.2 on Zeppelin branch-0.7
2. And Use JDK 8 for the spark interpreter.

这篇关于Zeppelin 0.7.2版本不支持spark 2.2.0的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！