在星火AWS EMR UI [英] Spark UI on AWS EMR

问题描述

我正在通过EMR控制台下拉装有火花AWS EMR集群（1.3.1）。 Spark是当前和处理数据，但我试图找到哪个端口已被分配给在WebUI。我试着用口无连接转发4040都和8080。我转发像这样

I am running a AWS EMR cluster with Spark (1.3.1) installed via the EMR console dropdown. Spark is current and processing data but I am trying to find which port has been assigned to the WebUI. I've tried port forwarding both 4040 and 8080 with no connection. I'm forwarding like so

ssh -i ~/KEY.pem -L 8080:localhost:8080 hadoop@EMR_DNS

1）如何找出星火WebUI中指定的端口是什么？
2）如何验证星火WebUI中运行？

1) How do I find out what the Spark WebUI's assigned port is? 2) How do I verify the Spark WebUI is running?

推荐答案

在电子病历星火配置为纱，从而星火UI是由YARN资源管理器提供的应用程序提供的网址（<一个href=\"http://spark.apache.org/docs/latest/monitoring.html\">http://spark.apache.org/docs/latest/monitoring.html).因此，要获得它的最简单的方法是设置与利用SSH从EMR控制台打开资源管理器打开了然后端口SOCKS，然后单击提供给运行的应用程序的正确应用主网址浏览器。星火历史服务器的可用默认端口18080.

Spark on EMR is configured for YARN, thus the Spark UI is available by the application url provided by the YARN Resource Manager (http://spark.apache.org/docs/latest/monitoring.html). So the easiest way to get to it is to setup your browser with SOCKS using a port opened by SSH then from the EMR console open Resource Manager and click the Application Master URL provided to the right of the running application. Spark History server is available at the default port 18080.

在<带EMR袜子为例href=\"http://docs.aws.amazon.com/ElasticMa$p$pduce/latest/DeveloperGuide/emr-web-interfaces.html\">http://docs.aws.amazon.com/ElasticMa$p$pduce/latest/DeveloperGuide/emr-web-interfaces.html

这篇关于在星火AWS EMR UI的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！