CDH5.2:MR，无法初始化任何输出收集器 [英] CDH5.2: MR, Unable to initialize any output collector
问题描述
Cloudera CDH5.2快速入门VM Cloudera Manager显示所有节点状态=绿色
Cloudera CDH5.2 Quickstart VM Cloudera Manager showing all nodes state = GREEN
我在Eclipse上执行了MR工作，包括构建路径中的所有相关cloudera jars: avro-1.7.6-cdh5.2.0.jar， avro-mapred-1.7.6-cdh5.2.0-hadoop2.jar， hadoop-common-2.5.0-cdh5.2.0.jar， hadoop-mapreduce-client-core-2.5.0-cdh5.2.0.jar
I've jared on Eclipse a MR job including all relevant cloudera jars in the Build Path: avro-1.7.6-cdh5.2.0.jar, avro-mapred-1.7.6-cdh5.2.0-hadoop2.jar, hadoop-common-2.5.0-cdh5.2.0.jar, hadoop-mapreduce-client-core-2.5.0-cdh5.2.0.jar
我已经完成了以下工作
hadoop jar jproject1.jar avro00.AvroUserPrefCount -libjars ${LIBJARS} avro/00/in avro/00/out
我收到以下错误，这是Java堆问题，有任何评论吗?预先谢谢你
I get the following error, is it a Java heap problem, any comments ? Thank you in advance
14/11/14 01:02:40 INFO client.RMProxy: Connecting to ResourceManager at quickstart.cloudera/127.0.0.1:8032
14/11/14 01:02:43 INFO input.FileInputFormat: Total input paths to process : 1
14/11/14 01:02:43 INFO mapreduce.JobSubmitter: number of splits:1
14/11/14 01:02:44 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1415950730849_0001
14/11/14 01:02:45 INFO impl.YarnClientImpl: Submitted application application_1415950730849_0001
14/11/14 01:02:45 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1415950730849_0001/
14/11/14 01:02:45 INFO mapreduce.Job: Running job: job_1415950730849_0001
14/11/14 01:03:04 INFO mapreduce.Job: Job job_1415950730849_0001 running in uber mode : false
14/11/14 01:03:04 INFO mapreduce.Job: map 0% reduce 0%
14/11/14 01:03:11 INFO mapreduce.Job: Task Id : attempt_1415950730849_0001_m_000000_0, Status : FAILED
Error: java.io.IOException: Unable to initialize any output collector
at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:412)
at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:695)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
...
...
推荐答案
检查失败尝试的完整任务日志attempt_1415950730849_0001_m_000000_0将有助于告诉您为什么遇到给定的异常.
Checking the full task log of the failed attempt attempt_1415950730849_0001_m_000000_0 will help tell why you ran into the given exception.
观察到此错误的最常见原因是作业中io.sort.mb的错误配置值.它的值不得大于(或大于)配置的映射任务堆大小，并且当前不得超过~2000 MB(Java数组最大大小).
The most common reason of observing such an error is a misconfigured value of io.sort.mb in your job. Its value must never be anywhere close to (or higher than) the configured map task heap size, and must also not currently exceed ~2000 MB (Java array maximum size).
最近还通过 MAPREDUCE-6194 .
这篇关于CDH5.2:MR，无法初始化任何输出收集器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
