如何在Windows虚拟环境中的Jupyter Notebook中使pyspark工作? [英] How do I get pyspark working in Jupyter Notebook in a virtual environment on Windows?

问题描述

我收到了可怕的异常:Java网关进程在发送其端口号之前已退出"错误，但是我已经遵循了所有可以找到的内容，但仍然无法正常工作.最糟糕的是，我发誓上周进行了此设置，但现在不知何故了.

I'm receiving the dreaded 'Exception: Java gateway process exited before sending its port number' error but I've followed everything I can find already and it's still not working. The worst thing is I swear this set up worked last week and somehow doesn't anymore.

我可以在命令行和虚拟环境之外的虚拟环境中完美运行pyspark(我正在使用Pipenv)，因此这一定与Jupyter Notebook有关.有人可以在Windows上解决此问题吗?

I can run pyspark perfectly fine in the virtual env from the command line and outside of the virutal environment (I'm using Pipenv) so it must be something to do with Jupyter Notebook. Has anyone solved this problem on Windows who can help me?

推荐答案

我从

I figured out a fix from here. My JAVA, SPARK_HOME, and HADOOP_HOME environment variables were configured properly but I added

PYSPARK_DRIVER_PYTHON = jupyter PYSPARK_DRIVER_PYTHON_OPTS =笔记本

PYSPARK_DRIVER_PYTHON = jupyter PYSPARK_DRIVER_PYTHON_OPTS = notebook

它现在正在工作.

这篇关于如何在Windows虚拟环境中的Jupyter Notebook中使pyspark工作?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！