Pyspark：例外：在向驱动程序发送端口号之前退出Java网关进程 [英] Pyspark: Exception: Java gateway process exited before sending the driver its port number

问题描述

我正试图在macbook air上运行pyspark。当我尝试启动时，我收到错误：

I'm trying to run pyspark on my macbook air. When i try starting it up I get the error:

Exception: Java gateway process exited before sending the driver its port number

当sc =启动时调用SparkContext（）时。我尝试运行以下命令：

when sc = SparkContext() is being called upon startup. I have tried running the following commands:

./bin/pyspark
./bin/spark-shell
export PYSPARK_SUBMIT_ARGS="--master local[2] pyspark-shell"

无效。我也看过这里：

Spark + Python - 在向驱动程序发送端口号之前退出Java网关进程？

但是问题从未得到解答。请帮忙！谢谢。

but the question has never been answered. Please help! Thanks.

推荐答案

这应该可以帮到你

一个解决方案是添加pyspark-shell到shell环境变量PYSPARK_SUBMIT_ARGS：

One solution is adding pyspark-shell to the shell environment variable PYSPARK_SUBMIT_ARGS:

export PYSPARK_SUBMIT_ARGS="--master local[2] pyspark-shell"

python / pyspark / java_gateway.py有一个变化，要求PYSPARK_SUBMIT_ARGS包括pyspark-shell PYSPARK_SUBMIT_ARGS变量由用户设置。

There is a change in python/pyspark/java_gateway.py , which requires PYSPARK_SUBMIT_ARGS includes pyspark-shell if a PYSPARK_SUBMIT_ARGS variable is set by a user.

这篇关于Pyspark：例外：在向驱动程序发送端口号之前退出Java网关进程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！