关闭并打开Goog​​le Cloud虚拟机(计算引擎)后，动态端口转发失败 [英] Dynamic port forwarding fails after turning off and on Google Cloud virtual machine (compute engine)

问题描述

我正在通过动态端口转发连接到我的Spark集群主节点，以便可以在本地计算机中打开jupyter笔记本Web界面.

I'm connecting to my Spark cluster master node with dynamic port forwarding so that I can open jupyter notebook web interface in my local machine.

我遵循了此Google Cloud Dataproc教程中的说明: https://cloud.google.com/dataproc/docs/tutorials/jupyter-notebook

I followed the instructions from this Google Cloud Dataproc tutorial: https://cloud.google.com/dataproc/docs/tutorials/jupyter-notebook

我根据建议使用以下命令创建了ssh漏斗:

I created ssh funnel with the following command as advised:

gcloud compute ssh --zone=<cluster-zone> --ssh-flag="-D" --ssh-flag="10000" --ssh-flag="-N" "<cluster-name>-m"

并打开了Web界面:

<browser executable path> \
"http://<cluster-name>-m:8123" \
--proxy-server="socks5://localhost:10000" \
--host-resolver-rules="MAP * 0.0.0.0 , EXCLUDE localhost" \
--user-data-dir=/tmp/

我第一次尝试时效果很好.

It worked perfectly fine first time I tried.

但是，一旦我关闭goole计算引擎并在一段时间后将其打开，完全相同的命令将不起作用，并在下面显示错误消息:

However, once I turned my goole compute engine off and turned it on after a while, the exact same commands doesn't work, giving out error message below:

debug1: Connection to port 10000 forwarding to socks port 0 requested.
debug2: fd 8 setting TCP_NODELAY
debug3: fd 8 is O_NONBLOCK
debug3: fd 8 is O_NONBLOCK
debug1: channel 2: new [dynamic-tcpip]
debug2: channel 2: pre_dynamic: have 0
debug2: channel 2: pre_dynamic: have 3
debug2: channel 2: decode socks5
debug2: channel 2: socks5 auth done
debug2: channel 2: pre_dynamic: need more
debug2: channel 2: pre_dynamic: have 0
debug2: channel 2: pre_dynamic: have 19
debug2: channel 2: decode socks5
debug2: channel 2: socks5 post auth
debug2: channel 2: dynamic request: socks5 host cluster-1-m port 8123 command 1
channel 2: open failed: connect failed: Connection refused
debug2: channel 2: zombie
debug2: channel 2: garbage collecting
debug1: channel 2: free: direct-tcpip: listening port 10000 for cluster-1-m port 8123, connect from ::1 port 49535 to ::1 port 10000, nchannels 3
debug3: channel 2: status: The following connections are open:

等待帮助:D

推荐答案

重新启动后，不会重新启动Jupyter笔记本内核.计算机启动后，您需要自己手动重启笔记本电脑，例如:

The Jupyter notebook kernel is not relaunched after reboots. You'll need to manually restart the notebook yourself once the machine has booted, e.g.:

gcloud compute ssh <cluster-name>-m
nohup /usr/local/bin/miniconda/bin/jupyter notebook --no-browser > /var/log/jupyter_notebook.log 2>&1 &

一旦内核启动并运行，您就应该能够通过代理访问Web UI.

Once the kernel is up and running, you should be able to access the web UI by proxy.

注意:通常，Dataproc不支持停止或重新启动整个集群.

Note: In general, Dataproc does not support stopping or restarting the entire cluster.

这篇关于关闭并打开Goog​​le Cloud虚拟机(计算引擎)后，动态端口转发失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！