使用Spark 2启动Ipython [英] Starting Ipython with Spark 2
问题描述
我的ipthon启动脚本如下：
I have my ipthon startup script as follows
IPYTHON_OPTS="notebook --port 8889 \
--notebook-dir='/usr/hdp/2.3.2.0-2950/spark/' \
--ip='*' --no-browser" pyspark
它适用于较旧的Spark。但是当我切换到Spark2时，我收到以下错误：
It runs fine for older Spark. But when I switch to Spark2, I get the following error:
在Spark 2.0+中删除了IPYTHON和IPYTHON_OPTS。从环境中删除它们并改为设置PYSPARK_DRIVER_PYTHON和PYSPARK_DRIVER_PYTHON_OPTS。
IPYTHON and IPYTHON_OPTS are removed in Spark 2.0+. Remove these from the environment and set PYSPARK_DRIVER_PYTHON and PYSPARK_DRIVER_PYTHON_OPTS instead.
目前尚不清楚应该用什么命令替换它。有人可以帮忙吗？
It is not clear what command I should replace it with. Can someone help?
推荐答案
对于Spark< 2.0使用IPython启动 pyspark 的命令是：
For Spark < 2.0 the command to start pyspark with IPython was:
IPYTHON = 1 pyspark
Spark> = 2.0的等效更新命令是：
The equivalent updated command for Spark >= 2.0 is:
PYSPARK_DRIVER_PYTHON = ipython pyspark
根据 Spark的源代码。
这篇关于使用Spark 2启动Ipython的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
