我如何以编程方式获取Hadoop在Web界面中显示的所有作业跟踪器和任务跟踪器信息？ [英] How could I programmatically get all the job tracker and tasktracker information that is displayed by Hadoop in the web interface?

问题描述

我使用Cloudera的Hadoop发行版CDH-0.20.2CDH3u0。
有没有什么办法可以获得诸如jobtracker状态，tasktracker状态，使用在hadoop框架之外运行的JAVA程序的计数器等信息？我尝试使用JMX进行监听，但hadoop提供有关Jobtracker，tasktracker和datanode的信息非常少。它不提供任何与运行作业状态相关的JMX属性，如地图完成百分比，减少完成百分比，完成任务百分比，尝试完成百分比，计数器状态等。我尝试使用由hadoop倾倒的指标日志。但它也没有包含关于map / reduce完成百分比，完成任务百分比的任何信息。

我认为，应该有一些替代方法来获得所有这些东西。

请回复。

解决方案

我不确定这是否是正确的，但你可以尝试HUE。我认为HUE提供了有关工作的信息。自开源以来，您可以看到他们如何访问作业跟踪器和名称跟踪器。

I'm using Cloudera's Hadoop distribution CDH-0.20.2CDH3u0. Is there any way I could the information such as jobtracker status, tasktracker status, counters using a JAVA program running outside of hadoop framework? I tried listening using JMX but hadoop provides very less information regarding Jobtracker, tasktracker and datanode. It doesn't provide any JMX attributes related to running job state like map percent completion, reduce percent completion, task percent completion, attempt percent completion, counters status etc.

Futhermore I tried using the metrics logs dumped by hadoop. But it too doesn't contain any information regarding map/reduce percent completion, task percent completion.

I think, there should be some alternative way to get all those stuffs.

Please do reply.

解决方案

I am not sure if this is correct but you can try HUE. I think HUE gives information about jobs. Since its open source you can see how they access job tracker and name tracker.

这篇关于我如何以编程方式获取Hadoop在Web界面中显示的所有作业跟踪器和任务跟踪器信息？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！