从机上的 DiskErrorException - Hadoop 多节点 [英] DiskErrorException on slave machine - Hadoop multinode

问题描述

我正在尝试从 hadoop 处理 XML 文件，但在对 XML 文件调用字数统计作业时出现以下错误.

I am trying to process XML files from hadoop, i got following error on invoking word-count job on XML files .

13/07/25 12:39:57 INFO mapred.JobClient: Task Id : attempt_201307251234_0001_m_000008_0, Status : FAILED
Too many fetch-failures
13/07/25 12:39:58 INFO mapred.JobClient:  map 99% reduce 0%
13/07/25 12:39:59 INFO mapred.JobClient:  map 100% reduce 0%
13/07/25 12:40:56 INFO mapred.JobClient: Task Id : attempt_201307251234_0001_m_000009_0, Status : FAILED
Too many fetch-failures
13/07/25 12:40:58 INFO mapred.JobClient:  map 99% reduce 0%
13/07/25 12:40:59 INFO mapred.JobClient:  map 100% reduce 0%
13/07/25 12:41:22 INFO mapred.JobClient:  map 100% reduce 1%
13/07/25 12:41:57 INFO mapred.JobClient: Task Id : attempt_201307251234_0001_m_000015_0, Status : FAILED
Too many fetch-failures
13/07/25 12:41:58 INFO mapred.JobClient:  map 99% reduce 1%
13/07/25 12:41:59 INFO mapred.JobClient:  map 100% reduce 1%
13/07/25 12:42:57 INFO mapred.JobClient: Task Id : attempt_201307251234_0001_m_000014_0, Status : FAILED
Too many fetch-failures
13/07/25 12:42:58 INFO mapred.JobClient:  map 99% reduce 1%
13/07/25 12:42:59 INFO mapred.JobClient:  map 100% reduce 1%
13/07/25 12:43:22 INFO mapred.JobClient:  map 100% reduce 2%

我观察者在从机上的 hadoop-hduser-tasktracker-localhost.localdomain.log 文件中出现错误.

i observer following error at hadoop-hduser-tasktracker-localhost.localdomain.log file on slave machine .

2013-07-25 12:38:58,124 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201307251234_0001_m_000001_0,0) failed :
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hduser/jobcache/job_201307251234_0001/attempt_201307251234_0001_m_000001_0/output/file.out.index in any of the configured local directories
        at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)

当我运行文本文件时，这很好用

This works fine when i ran for text files

推荐答案

看起来你已经点击了这个 问题.应用补丁或下载固定版本，你应该很高兴.

Looks like you have hit this issue. Either apply the patch or download the fixed version, and you should be good to go.

HTH

这篇关于从机上的 DiskErrorException - Hadoop 多节点的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！