火花工作者未连接到主设备 [英] spark worker not connecting to master

问题描述

我想创建一个独立的Spark集群.我可以在同一节点上运行主服务器和从属服务器，但是不同节点上的从属服务器既不显示master-URL也不连接到主服务器.

I want to create a spark standalone cluster. I am able to run master and slave on same node, but the slave on different node is neither showing master-URL nor connecting to master.

我正在运行命令:

start-slave.sh spark://spark-server:7077

其中spark-server是我的主服务器的主机名.

where spark-server is the hostname of my master.

我能够从worker ping主机，但是master的WebUI除了在同一台计算机上运行之外，没有显示任何worker.客户端节点正在运行工作服务器，但它是独立的，并且未连接到主服务器.

I am able to ping master from worker, but the WebUI of master isn't showing any worker except that running on same machine. The client node is running a worker but it is independent and not connected to the master.

推荐答案

请检查主节点上的配置文件" spark-env.sh ".您是否已将 SPARK_MASTER_HOST 变量设置为主节点的IP地址?如果没有，请尝试设置它并重新启动主服务器和从属服务器.例如，如果主节点的IP为192.168.0.1，则其中应为 SPARK_MASTER_HOST = 192.168.0.1 .请注意，您无需在从站上设置此变量.

Please check configuration file "spark-env.sh" on your master node. Have you set the SPARK_MASTER_HOST variable to the IP address of the master node? If not try to set it and restart the master and slaves. For example, if your master node's IP is 192.168.0.1, you should have SPARK_MASTER_HOST=192.168.0.1 in there. Note that you don't need to set this variable on your slaves.

这篇关于火花工作者未连接到主设备的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！