在 Windows 上使用 pyspark 不起作用 - py4j [英] Using pyspark on Windows not working- py4j

问题描述

我使用 在 Windows 上安装了 Zeppelin本教程和this.我还安装了 java 8 以避免出现问题.

I installed Zeppelin on Windows using this tutorial and this. I also installed java 8 to avoid problems.

我现在可以启动 Zeppelin 服务器，并且我正在尝试运行此代码 -

I'm now able to start the Zeppelin server, and I'm trying to run this code -

%pyspark
a=5*4
print("value = %i" % (a))
sc.version

我收到此错误，与 py4j 相关.我之前遇到过这个库的其他问题(与 此处 相同)，为了避免它们，我将计算机上 Zeppelin 和 Spark 中的 py4j 库替换为最新版本 - py4j 0.10.7.

I'm getting this error, related to py4j. I had other problems with this library before (same as here), and to avoid them I replaced the library of py4j in the Zeppelin and Spark on my computer with the latest version- py4j 0.10.7.

这是我得到的错误-

Traceback (most recent call last):
  File "C:\Users\SHIRM~1.ARG\AppData\Local\Temp\zeppelin_pyspark-1240802621138907911.py", line 309, in <module>
    sc = _zsc_ = SparkContext(jsc=jsc, gateway=gateway, conf=conf)
  File "C:\Users\SHIRM.ARGUS\spark-2.3.2\spark-2.3.2-bin-hadoop2.7\python\pyspark\context.py", line 118, in __init__
    conf, jsc, profiler_cls)
  File "C:\Users\SHIRM.ARGUS\spark-2.3.2\spark-2.3.2-bin-hadoop2.7\python\pyspark\context.py", line 189, in _do_init
    self._javaAccumulator = self._jvm.PythonAccumulatorV2(host, port, auth_token)
  File "C:\Users\SHIRM.ARGUS\Documents\zeppelin-0.8.0-bin-all\interpreter\spark\pyspark\py4j-0.10.7-src.zip\py4j\java_gateway.py", line 1525, in __call__
  File "C:\Users\SHIRM.ARGUS\Documents\zeppelin-0.8.0-bin-all\interpreter\spark\pyspark\py4j-0.10.7-src.zip\py4j\protocol.py", line 332, in get_return_value
py4j.protocol.Py4JError: An error occurred while calling None.org.apache.spark.api.python.PythonAccumulatorV2. Trace:

我用谷歌搜索了它，但找不到发生在它身上的任何人.

I googled it, but couldn't find anyone that it had happened to.

有人知道我该如何解决这个问题吗?

Does anyone have an idea how can I solve this?

谢谢

推荐答案

我觉得你已经安装了 Java 9 或 10.卸载其中一个版本并从这里安装 Java 8 的新副本:https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

I feel you have installed Java 9 or 10. Uninstall either of those versions and install a fresh copy of Java 8 from here: https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

并且在 hadoop_env.cmd 中设置 JAVA_HOME(使用任何文本编辑器打开).

注意:Java 8 或 7 是稳定版本，可以使用和卸载任何现有版本的 JAVA.确保在 JAVA_HOME 中添加 JDK(不是 JRE).

Note: Java 8 or 7 are stable versions to use and uninstall any existing versions of JAVA. Make sure you add JDK (not JRE) in JAVA_HOME.

这篇关于在 Windows 上使用 pyspark 不起作用 - py4j的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！