没有在Windows上安装Apache Spark的名为“资源"的模块 [英] No module named 'resource' installing Apache Spark on Windows

问题描述

我正在尝试安装apache spark，以便在Windows机器上本地运行.我已按照此处的所有说明进行操作

I am trying to install apache spark to run locally on my windows machine. I have followed all instructions here https://medium.com/@loldja/installing-apache-spark-pyspark-the-missing-quick-start-guide-for-windows-ad81702ba62d.

安装后，我能够成功启动pyspark并执行诸如

After this installation I am able to successfully start pyspark, and execute a command such as

textFile = sc.textFile("README.md")

然后我执行对textFile等操作的命令时，例如

When I then execute a command that operates on textFile such as

textFile.first()

Spark给我错误工人无法重新连接"，并且我可以在控制台中看到来自worker.py的异常，说"ModuleNotFoundError:没有名为资源的模块".查看源文件，我可以看到该python文件确实确实尝试导入资源模块，但是该模块在Windows系统上不可用.我了解您可以在Windows上安装spark，所以如何解决这个问题?

Spark gives me the error 'worker failed to connect back', and I can see an exception in the console coming from worker.py saying 'ModuleNotFoundError: No module named resource'. Looking at the source file I can see that this python file does indeed try to import the resource module, however this module is not available on windows systems. I understand that you can install spark on windows so how do I get around this?

推荐答案

我整个上午都在为同样的问题而苦苦挣扎.最好的选择是降级到Spark 2.3.2

I struggled the whole morning with the same problem. Your best bet is to downgrade to Spark 2.3.2

这篇关于没有在Windows上安装Apache Spark的名为“资源"的模块的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！