无法运行 pyspark [英] Unable to run pyspark

问题描述

我在 Windows 上安装了 Spark，但无法启动 pyspark.当我输入 c:\Spark\bin\pyspark 时，出现以下错误:

I installed Spark on Windows, and I'm unable to start pyspark. When I type in c:\Spark\bin\pyspark, I get the following error:

Python 3.6.0 |Anaconda 自定义(64 位)|(默认，2016 年 12 月 23 日，11:57:41)win32 上的 [MSC v.1900 64 位 (AMD64)] 输入帮助"、版权"、信用"或许可证"以获取更多信息.回溯(最近一次调用):文件c:\Spark\bin..\python\pyspark\shell.py"，第 30 行，导入 pyspark 文件c:\Spark\python\pyspark__init__.py"，第 44 行, in from pyspark.context import SparkContext File "c:\Spark\python\pyspark\context.py", line 36, in from pyspark.java_gateway import launch_gateway File "c:\Spark\python\pyspark\java_gateway.py",第 31 行，从 py4j.java_gateway 导入 java_import，JavaGateway，GatewayClient 文件"，第 961 行，在 _find_and_load 文件"，第 950 行，在 _find_and_load_unlocked 文件"，第 646 行，在 _load_unlocked 文件"，第 616 行，load_back 兼容_文件c:\Spark\python\lib\py4j-0.10.4-src.zip\py4j\java_gateway.py"，第 18 行，在文件C:\Users\Eigenaar\Anaconda3\lib\pydoc.py"中，第 62 行，在 import pkgutil File "C:\Users\Eigenaar\Anaconda3\lib\pkgutil.py"，第 22 行，在 ModuleInfo = namedtuple('ModuleInfo', 'module_finder name ispkg') 文件 "c:\Spark\python\pyspark\serializers.py"，第 393 行，在 namedtuple cls = _old_namedtuple(*args, **kwargs) 类型错误:namedtuple() 缺少 3 个必需的仅关键字参数:verbose"、rename"和module"

Python 3.6.0 |Anaconda custom (64-bit)| (default, Dec 23 2016, 11:57:41) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. Traceback (most recent call last): File "c:\Spark\bin..\python\pyspark\shell.py", line 30, in import pyspark File "c:\Spark\python\pyspark__init__.py", line 44, in from pyspark.context import SparkContext File "c:\Spark\python\pyspark\context.py", line 36, in from pyspark.java_gateway import launch_gateway File "c:\Spark\python\pyspark\java_gateway.py", line 31, in from py4j.java_gateway import java_import, JavaGateway, GatewayClient File "", line 961, in _find_and_load File "", line 950, in _find_and_load_unlocked File "", line 646, in _load_unlocked File "", line 616, in _load_backward_compatible File "c:\Spark\python\lib\py4j-0.10.4-src.zip\py4j\java_gateway.py", line 18, in File "C:\Users\Eigenaar\Anaconda3\lib\pydoc.py", line 62, in import pkgutil File "C:\Users\Eigenaar\Anaconda3\lib\pkgutil.py", line 22, in ModuleInfo = namedtuple('ModuleInfo', 'module_finder name ispkg') File "c:\Spark\python\pyspark\serializers.py", line 393, in namedtuple cls = _old_namedtuple(*args, **kwargs) TypeError: namedtuple() missing 3 required keyword-only arguments: 'verbose', 'rename', and 'module'

我在这里做错了什么?

推荐答案

Spark 2.1.0 不支持 python 3.6.0.要解决此问题，请在 anaconda 环境中更改您的 Python 版本.在 anaconda env 中运行以下命令

Spark 2.1.0 doesn't support python 3.6.0. To solve this change your python version in anaconda environment. Run following command in your anaconda env

conda create -n py35 python=3.5 anaconda
activate py35

这篇关于无法运行 pyspark的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！