无法连接与谷歌Compute Engine的星火掌握 [英] Fail to connect to master with Spark on Google Compute Engine

问题描述

我想通过在谷歌计算引擎的Hadoop /火花集群启动单击部署的软件功能。

I am trying hadoop/spark cluster in Google Compute Engine through "Launch click-to-deploy software" feature .

我已经创建了1主2从节点，我可以在群集上推出火花外壳，但是当我想从我的电脑发动火花壳，我失败了。

I have created 1 master and 2 slave node and i can launch spark-shell on the cluster but when i want to launch spark-shell since my computer, i failed.

我启动：

./bin/spark-shell --master spark://IP or Hostname:7077

和我有这个堆栈跟踪：

15/04/09 10:58:06 INFO AppClient$ClientActor: Connecting to master
akka.tcp://sparkMaster@IP or Hostname:7077/user/Master...
15/04/09 10:58:06 WARN AppClient$ClientActor: Could not connect to
akka.tcp://sparkMaster@IP or Hostname:7077: akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@IP or Hostname:7077
15/04/09 10:58:06 WARN Remoting: Tried to associate with unreachable remote address [akka.tcp://sparkMaster@IP or Hostname:7077]. Address is now gated for 5000 ms, all messages to this address will be delivered to dead letters. Reason: IP or Hostname: unknown error

请让我知道如何解决这个问题。

please let me know how to overcome this problem .

推荐答案

请参阅丹尼尔Darabos评论。默认情况下，被封锁除了SSH，RDP和ICMP所有传入连接。为了能够从互联网到Hadoop的主实例连接，则必须打开端口7077的Hadoop的硕士项目中的第一个标签：

See comment from Daniel Darabos. By default, all incoming connections are blocked except for SSH, RDP and ICMP. To be able to connect from the Internet to the hadoop master instance, you must open port 7077 for 'hadoop-master' tag in your project first:

gcloud compute --project PROJECT firewall-rules create allow-spark \
    --allow TCP:7077 \
    --target-tags hadoop-master

请参阅防火墙，的添加防火墙和的 gcloud计算的防火墙规则在进一步的细节，并做好一切准备GCE公开文件制作。

这篇关于无法连接与谷歌Compute Engine的星火掌握的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！