Spark主机:7077无法访问 [英] Spark master-machine:7077 not reachable

问题描述

我有一个Spark Spark集群，其中主节点也是工作节点.我无法从驱动程序代码节点到达主服务器，并且出现错误:

I have a Spark Spark cluster where the master node is also the worker node. I can't reach the master from the driver-code node, and I get the error:

14:07:10 WARN client.AppClient$ClientEndpoint: Failed to connect to master master-machine:7077

驱动程序代码节点中的SparkContext配置为: SparkConf conf = new SparkConf(true).setMaster(spark:master-machine//:7077);

The SparkContext in driver-code node is configured as: SparkConf conf = new SparkConf(true).setMaster(spark:master-machine//:7077);

我可以成功地ping master-machine，但是我不能成功地telnet master-machine 7077.表示机器可以访问，但端口不可达.

I can successfully ping master-machine, but I can't successfully telnet master-machine 7077. Meaning the machine is reachable but the port is not.

可能是什么问题?我已经为主节点和运行驱动程序代码的节点(客户端)禁用了Ubuntu的ufw防火墙.

What could be the issue? I have disabled Ubuntu's ufw firewall for both master node and node where driver code runs (client).

推荐答案

您的语法有些偏离，您可以:

Your syntax is a bit off, you have:

setMaster(spark:master-machine//:7077)

您要:

setMaster(spark://master-machine:7077)

从Spark文档中:

启动后，母版将打印出spark://HOST:PORT URL 本身，您可以使用它来连接工作人员或通过 SparkContext的主"参数.您也可以在 大师的网络用户界面，默认情况下为http://localhost:8080.

Once started, the master will print out a spark://HOST:PORT URL for itself, which you can use to connect workers to it, or pass as the "master" argument to SparkContext. You can also find this URL on the master’s web UI, which is http://localhost:8080 by default.

您也可以在其中使用IP地址，在我总是必须使用IP地址的基于debian的安装中遇到了问题，但这是一个单独的问题.一个例子:

You can use an IP address in there too, I have run into issues with debian-based installs where I always have to use the IP address but that's a separate issue. An example:

spark.master            spark://5.6.7.8:7077

在Spark文档中的配置页面

From a configuration page in Spark docs

这篇关于Spark主机:7077无法访问的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！