Worker:无法连接到 spark apache 上的 master [英] Worker: Failed to connect to master on spark apache

问题描述

我正在尝试在 Windows 上使用独立集群管理器部署 Spark 应用程序，因此在主端(IP:192...186)我运行:

I am trying to deploy a Spark application using standalone cluster manager on windows, so in the master side (IP: 192...186) I ran:

\bin>spark-class org.apache.spark.deploy.master.Master

在从端(IP:192...75)我跑了:

and in the slave side (IP: 192...75) I ran:

\bin>spark-class org.apache.spark.deploy.worker.Worker spark://192.*.*.186:7077

我有一个问题，客户端无法连接到主节点，这是我得到的错误:

I have a problem, the client can not connect to the master node and this is the error that I am getting:

17/09/26 12:00:51 WARN Worker: Failed to connect to master 192.*.*.186:7077
org.apache.spark.SparkException: Exception thrown in awaitResult:
    at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:205)
    at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:75)
    at org.apache.spark.rpc.RpcEnv.setupEndpointRefByURI(RpcEnv.scala:100)
    at org.apache.spark.rpc.RpcEnv.setupEndpointRef(RpcEnv.scala:108)
    at org.apache.spark.deploy.worker.Worker$$anonfun$org$apache$spark$deploy$worker$Worker$$tryRegisterAllMasters$1$$anon$1.run(Worker.scala:241)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Failed to connect to /192.*.*.186:7077
    at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:232)
    at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:182)
    at org.apache.spark.rpc.netty.NettyRpcEnv.createClient(NettyRpcEnv.scala:197)
    at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:194)
    at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:190)
    ... 4 more
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: /192.*.*.186:7077
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
    at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:257)
    at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:291)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:631)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:566)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:480)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:442)
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
    ... 1 more

我该如何解决这个问题?

how can I fix this?

推荐答案

bin>spark-class org.apache.spark.deploy.worker.Worker spark://192.*.*.186:7077

上面命令中提到的master-ip是否正确?

Is the master-ip mentioned in above command correct ?

您写的主 IP 是 192...168，但在从属启动命令中，您正在尝试使用 192...186，错字:6 和 8 倒置

You wrote that master ip is 192...168 but in slave start command you are trying with 192...186 , typo: 6 and 8 inverted

这篇关于Worker:无法连接到 spark apache 上的 master的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！