如何以编程方式获取 Hadoop 在 Web 界面中显示的所有作业跟踪器和任务跟踪器信息? [英] How could I programmatically get all the job tracker and tasktracker information that is displayed by Hadoop in the web interface?

问题描述

我正在使用 Cloudera 的 Hadoop 发行版 CDH-0.20.2CDH3u0.有什么方法可以使用在 hadoop 框架之外运行的 JAVA 程序来获取诸如作业跟踪器状态、任务跟踪器状态、计数器之类的信息?我尝试使用 JMX 进行监听，但 hadoop 提供的有关 Jobtracker、tasktracker 和 datanode 的信息非常少.它不提供任何与运行作业状态相关的 JMX 属性，例如映射完成百分比、减少完成百分比、任务完成百分比、尝试完成百分比、计数器状态等.

I'm using Cloudera's Hadoop distribution CDH-0.20.2CDH3u0. Is there any way I could the information such as jobtracker status, tasktracker status, counters using a JAVA program running outside of hadoop framework? I tried listening using JMX but hadoop provides very less information regarding Jobtracker, tasktracker and datanode. It doesn't provide any JMX attributes related to running job state like map percent completion, reduce percent completion, task percent completion, attempt percent completion, counters status etc.

此外，我尝试使用 hadoop 转储的指标日志.但它也不包含任何关于 map/reduce 完成百分比、任务完成百分比的信息.

Futhermore I tried using the metrics logs dumped by hadoop. But it too doesn't contain any information regarding map/reduce percent completion, task percent completion.

我认为，应该有一些替代方法来获取所有这些东西.

I think, there should be some alternative way to get all those stuffs.

请回复.

推荐答案

我不确定这是否正确，但您可以尝试 HUE.我认为 HUE 提供有关工作的信息.由于它是开源的，您可以看到他们如何访问工作跟踪器和名称跟踪器.

I am not sure if this is correct but you can try HUE. I think HUE gives information about jobs. Since its open source you can see how they access job tracker and name tracker.

这篇关于如何以编程方式获取 Hadoop 在 Web 界面中显示的所有作业跟踪器和任务跟踪器信息?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！