有没有所有标准（非自定义）Hadoop mapreduce计数器的列表？ [英] Is there a list of all standard (not custom) Hadoop mapreduce counters?

问题描述

是否有可用的计数器列表可以从 [RunningJob] [1] 中使用方法 getCouters（）？

Is there list of available counters that can be retrieved from [RunningJob][1] using method getCouters()?

第二个问题：有没有计数器告诉我这个工作要执行多长时间？

Second question: Is there any counter that tells me how long did it take for the job to execute?

推荐答案

计数器列表

---

源

List of Counters

---

Source

Group Name                                  Counter Name

---

---

org.apache.hadoop.mapred.Task$Counter       MAP_INPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter       MAP_OUTPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter       MAP_SKIPPED_RECORDS
org.apache.hadoop.mapred.Task$Counter       MAP_INPUT_BYTES
org.apache.hadoop.mapred.Task$Counter       MAP_OUTPUT_BYTES
org.apache.hadoop.mapred.Task$Counter       COMBINE_INPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter       COMBINE_OUTPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter       REDUCE_INPUT_GROUPS
org.apache.hadoop.mapred.Task$Counter       REDUCE_SHUFFLE_BYTES
org.apache.hadoop.mapred.Task$Counter       REDUCE_INPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter       REDUCE_OUTPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter       REDUCE_SKIPPED_GROUPS
org.apache.hadoop.mapred.Task$Counter       REDUCE_SKIPPED_RECORDS
org.apache.hadoop.mapred.JobInProgress$Counter  TOTAL_LAUNCHED_MAPS
org.apache.hadoop.mapred.JobInProgress$Counter  RACK_LOCAL_MAPS
org.apache.hadoop.mapred.JobInProgress$Counter  DATA_LOCAL_MAPS
org.apache.hadoop.mapred.JobInProgress$Counter  TOTAL_LAUNCHED_REDUCES
FileSystemCounters                                  FILE_BYTES_READ
FileSystemCounters                                  HDFS_BYTES_READ
FileSystemCounters                                  FILE_BYTES_WRITTEN
FileSystemCounters                                  HDFS_BYTES_WRITTEN

---

---

如何通过API获取计数器？：

---

---

How to get counter By API ?:

Ans

新API： context.getCounter （groupName，counterName）;

旧API： Reporter.getCounter（groupName，counterName）;

Qus：是否任何一个告诉我工作执行多长时间的计数器？

答案：是的，你可以看到你的JobTracker /终端

Ans : Yes you can see that info in your Jobtracker/Terminal

Submit Host Address: xx.xx.xxx.xxx
Job-ACLs: All users are allowed
Job Setup: Successful
Status: Succeeded
Started at: Wed Nov 28 16:29:06 IST 2012
Finished at: Wed Nov 28 16:29:53 IST 2012
Finished in: 47sec
Job Cleanup: Successful

这篇关于有没有所有标准（非自定义）Hadoop mapreduce计数器的列表？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！