如何获取映射器输出字节计数器 [英] How to get the mapper output byte counter

问题描述

我要为并行BFS最短路径算法将数量不确定的map reduce作业链接在一起，当无法确定路径时，我的作业将无限循环而不会产生任何记录。我认为最好的检查方法是获取hadoop维护的Map Output Bytes计数器。

I am chaining an undetermined amount of map reduce jobs together for a parallel BFS shortest path algorithm and when the path cannot be determined, my jobs loop infinitely without producing any records. I figured the best way to check this is to get the Map Output Bytes counter that is maintained by hadoop.

如何访问此计数器？

推荐答案

到获取作业产生的地图输出字节计数器，请使用

To get the map output bytes counter produced by the job, use

long outputBytes = job.getCounters().findCounter("org.apache.hadoop.mapred.Task$Counter", "MAP_OUTPUT_BYTES").getValue();

请参见 http://lintool.github.io/Cloud9/docs/content/counters.html 了解更多的计数器组和名称

See http://lintool.github.io/Cloud9/docs/content/counters.html for more counter groups and names

这篇关于如何获取映射器输出字节计数器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！