如何通过API访问Hadoop计数器值？ [英] How to access Hadoop counters values via API?

问题描述

在Hadoop中，我们可以在map / reduce任务中增加计数器，如下所示：

  ... 
 context.getCounter（MyCountersEnum.SomeCounter）.increment（1）; 
 ... 
  

您可以在log中找到它们的值。

如何在作业完成后通过代码访问它们？

Hadoop API读取计数器值的方法是什么？

解决方案

我刚刚找到答案 here 。

您需要一个作业对象才能访问计数器：

  Counters counters = job.getCounters（）; 
 Counter counter = counters.findCounter（MyCountersEnum.SomeCounter）; 
 System.out.println（counter.getDisplayName（）+：+ counter.getValue（））; 
  

In Hadoop we can increment counter in map/reduce task, it looks like this:

...
context.getCounter(MyCountersEnum.SomeCounter).increment(1);
...

Than you can find their value in log.

How do you access them from code after job completes?

What is Hadoop API to read counter value?

解决方案

I just found the answer here.

You need a job object to access the counters:

Counters counters = job.getCounters();
Counter counter = counters.findCounter(MyCountersEnum.SomeCounter);
System.out.println(counter.getDisplayName() + ": " + counter.getValue());

这篇关于如何通过API访问Hadoop计数器值？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！