Hadoop wordcount无法运行 - 在解码hadoop错误消息时需要帮助 [英] Hadoop wordcount unable to run - need help on decoding the hadoop error message
本文介绍了Hadoop wordcount无法运行 - 在解码hadoop错误消息时需要帮助的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
节点集群来试用它。我遵循这里。
一切似乎都正常工作。我格式化了namenode，并且可以通过Web界面连接到jobtracker，datanode和namenode。我可以启动和停止所有hadoop服务。
但是，当我尝试运行 wordcount 示例时，我得到这个：
错误初始化attempt_201105161023_0002_m_000011_0：
java.io.IOException：读取文件异常：/ app / hadoop / tmp / mapred / local / ttprivate / taskTracker / hadoop / jobcache / job_201105161023_0002 / jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile（Credentials.java:135）
at org.apache .hadoop.mapreduce.security.TokenCache.loadTokens（TokenCache.java:163）
at org.apache.hadoop.mapred.TaskTracker.initializeJob（TaskTracker.java:1064）
at org.apache.hadoop .mapred.TaskTracker.localizeJob（TaskTracker.java:1001）
at org.apache.hadoop.mapred.TaskTracker.startNewTask（TaskTracker.java:2161）
at org.apache.hadoop.mapred.TaskTracker $ TaskLauncher.run（TaskTracker.java:2125）
导致：java.io.FileNotFoundException：文件文件：/ app / hadoop / tmp / mapred / local / ttpriva te / taskTracker / hadoop / jobcache / job_201105161023_0002 / jobToken不存在。
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus（RawLocalFileSystem.java:371）
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus（FilterFileSystem.java:245）
（org.apache.hadoop.fs.ChecksumFileSystem $ ChecksumFSInputChecker。< init>（ChecksumFileSystem.java:125）
at org.apache.hadoop.fs.ChecksumFileSystem.open（ChecksumFileSystem.java:283）
在org.apache.hadoop.fs.FileSystem.open（FileSystem.java:400）
在org.apache.hadoop.security.Credentials.readTokenStorageFile（Credentials.java:129）
... 5 more
我在本地文件系统上创建了目录。
$ sudo mkdir / app / hadoop / tmp
$ sudo chown hadoop：hadoop / app / hadoop / tmp
我还修改了 core-site.xml ：
<！ - In：conf / core-site.xml - >
<属性>
< name> hadoop.tmp.dir< / name>
< value> / app / hadoop / tmp< / value>
< description>其他临时目录的基础。< / description>
< / property>
格式化namenode时，它会成功创建本地
和HDFS上的子目录。
当我查看失败的 wordcount 输出的结果时，错误
消息抱怨I / O错误，在文件
/应用/ hadoop的/ TMP / mapred /本地/ ttprivate /的TaskTracker / hadoop的/ jobcache /
job_201105161023_0002 / jobToken
我做了一些故障排除，可以浏览
本地文件系统上的 jobToken 文件，问题。内容类似于 HDTS MapReduce.job 201105161023_0002
这是一个许可问题吗？我让hadoop进程用户能够写入所有子目录，并且能够创建该文件。那么还有什么可能是错的？
命令行中的一些更详细的输出：
hadoop @ jonathan-VirtualBox：/usr/local/hadoop/hadoop-0.20.203.0$ bin / hadoop jar hadoop-examples-0.20.203.0.jar wordcount app / download app / output4
11 / 05/16 13:38:56 INFO input.FileInputFormat：要输入的总输入路径：3
11/05/16 13:39:05信息mapred.JobClient：正在运行的作业：job_201105161222_0003
11/05 / 16 13点39分06秒INFO mapred.JobClient：地图0％减少0％
16年11月5日13时39分十七秒INFO mapred.JobClient：任务标识：attempt_201105161222_0003_m_000004_0，状态：FAILED
错误初始化attempt_201105161222_0003_m_000004_0：
java.io.IOException：异常读取文件：/ app / hadoop / tmp / mapred / local / ttprivate / taskTracker / hadoop / jobcache / job_201105161222_0003 / jobToken $ b $ org.apache.hadoop。 security.Credentials.readTokenStorageFile（Credentials.java:135）
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens（TokenCache.java:163）在org.apache.hadoop.mapred.TaskTracker.initializeJob（TaskTracker.java:1064）
在org.apache.hadoop.mapred.TaskTracker.localizeJob（TaskTracker.java:1001）
在org.apache.hadoop.mapred.TaskTracker.startNewTask（TaskTracker.java:2161）
在org.apache.hadoop.mapred.TaskTracker $ TaskLauncher.run（TaskTracker.java:2125）
引起：java.io.FileNotFoundException：文件文件：/ app / hadoop / tmp / mapred / local / ttprivate / taskTracker / hadoop / jobcache / job_201105161222_0003 / jobToken不存在。
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus（RawLocalFileSystem.java:371）
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus（FilterFileSystem.java:245）
（org.apache.hadoop.fs.ChecksumFileSystem $ ChecksumFSInputChecker。< init>（ChecksumFileSystem.java:125）
at org.apache.hadoop.fs.ChecksumFileSystem.open（ChecksumFileSystem.java:283）
在org.apache.hadoop.fs.FileSystem.open（FileSystem.java:400）
在org.apache.hadoop.security.Credentials.readTokenStorageFile（Credentials.java:129）
... 5 more
$ b $ 11/05/16 13:39:21警告mapred.JobClient：读取任务输出出错http：// jonathan-VirtualBox：50060 / tasklog？plaintext = true& tryptid = attempt_201105161222_0003_m_000004_0& filter = stdout
11/05/16 13:39:21 WARR mapred.JobClient：读取任务输出出错http：// jonathan-VirtualBox：50060 / tasklog？plaintext = true& attemptptid = attempt_201105161222_0003_m_000004_0& filter = stderr
11/05/16 13:39:22信息m apred.JobClient：任务ID：attempt_201105161222_0003_m_000004_1，状态：FAILED
初始化attempt_201105161222_0003_m_000004_1错误：
java.io.IOException：异常读取文件：/ app / hadoop / tmp / mapred / local / ttprivate / taskTracker / hadoop / jobcache / job_201105161222_0003 / jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile（Credentials.java:135）
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens（TokenCache。 Java的：在org.apache.hadoop.mapred.TaskTracker.initializeJob（TaskTracker.java:1064 163）
）
在org.apache.hadoop.mapred.TaskTracker.localizeJob（TaskTracker.java:1001）
at org.apache.hadoop.mapred.TaskTracker.startNewTask（TaskTracker.java:2161）
at org.apache.hadoop.mapred.TaskTracker $ TaskLauncher.run（TaskTracker.java:2125）$ b java.io.FileNotFoundException：$ b所致文件文件：/应用/ Hadoop的/ tmp目录/ mapred /本地/ ttprivate /的TaskTracker / Hadoop的/ jobcache / job_201105161222_0003 / jobToken不存在。
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus（RawLocalFileSystem.java:371）
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus（FilterFileSystem.java:245）
（org.apache.hadoop.fs.ChecksumFileSystem $ ChecksumFSInputChecker。< init>（ChecksumFileSystem.java:125）
at org.apache.hadoop.fs.ChecksumFileSystem.open（ChecksumFileSystem.java:283）
在org.apache.hadoop.fs.FileSystem.open（FileSystem.java:400）
在org.apache.hadoop.security.Credentials.readTokenStorageFile（Credentials.java:129）
... 5 more
$ b $ 11/05/16 13:39:22 WARN mapred.JobClient：读取任务输出出错http：// jonathan-VirtualBox：50060 / tasklog？plaintext = true& tryptid = attempt_201105161222_0003_m_000004_1& filter = stdout
11/05/16 13:39:22 WARN mapred.JobClient：读取任务输出错误http：// jonathan-VirtualBox：50060 / tasklog？plaintext = true& attemptptid = attempt_201105161222_0003_m_000004_1& filter = stderr
11/05/16 13:39:23 INFO m apred.JobClient：任务ID：attempt_201105161222_0003_m_000004_2，状态：FAILED
初始化attempt_201105161222_0003_m_000004_2错误：
java.io.IOException：读取文件异常：/ app / hadoop / tmp / mapred / local / ttprivate / taskTracker / hadoop / jobcache / job_201105161222_0003 / jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile（Credentials.java:135）
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens（TokenCache。 Java的：在org.apache.hadoop.mapred.TaskTracker.initializeJob（TaskTracker.java:1064 163）
）
在org.apache.hadoop.mapred.TaskTracker.localizeJob（TaskTracker.java:1001）在org.apache.hadoop.mapred.TaskTracker.startNewTask（TaskTracker.java:2161）
。在org.apache.hadoop.mapred.TaskTracker $ TaskLauncher.run（TaskTracker.java:2125）
导致：java.io.FileNotFoundException：文件文件：/ app / hadoop / tmp / mapred / local / ttprivate / taskTracker / hadoop / jobcache / job_201105161222_0003 / jobToken不存在。
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus（RawLocalFileSystem.java:371）
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus（FilterFileSystem.java:245）
（org.apache.hadoop.fs.ChecksumFileSystem $ ChecksumFSInputChecker。< init>（ChecksumFileSystem.java:125）
at org.apache.hadoop.fs.ChecksumFileSystem.open（ChecksumFileSystem.java:283）
在org.apache.hadoop.fs.FileSystem.open（FileSystem.java:400）
在org.apache.hadoop.security.Credentials.readTokenStorageFile（Credentials.java:129）
... 5更多
11/05/16 13:39:23警告mapred.JobClient：读取任务输出时出错http：// jonathan-VirtualBox：50060 / tasklog？plaintext = true& tryptid = attempt_201105161222_0003_m_000004_2& filter = stdout
11/05/16 13:39:23警告mapred.JobClient：读取任务输出错误http：// jonathan-VirtualBox：50060 / tasklog？plaintext = true& attemptptid = attempt_201105161222_0003_m_000004_2& filter = stderr
11/05/16 13:39:30信息m apred.JobClient：任务ID：attempt_201105161222_0003_m_000003_0，状态：FAILED
初始化try_201105161222_0003_m_000003_0：
错误java.io.IOException：读取文件异常：/ app / hadoop / tmp / mapred / local / ttprivate / taskTracker / hadoop / jobcache / job_201105161222_0003 / jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile（Credentials.java:135）
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens（TokenCache。 Java的：在org.apache.hadoop.mapred.TaskTracker.initializeJob（TaskTracker.java:1064 163）
）
在org.apache.hadoop.mapred.TaskTracker.localizeJob（TaskTracker.java:1001）在org.apache.hadoop.mapred.TaskTracker.startNewTask（TaskTracker.java:2161）
。在org.apache.hadoop.mapred.TaskTracker $ TaskLauncher.run（TaskTracker.java:2125）$ b java.io.FileNotFoundException：$ b所致文件文件：/应用/ Hadoop的/ tmp目录/ mapred /本地/ ttprivate /的TaskTracker / Hadoop的/ jobcache / job_201105161222_0003 / jobToken不存在。
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus（RawLocalFileSystem.java:371）
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus（FilterFileSystem.java:245）
（org.apache.hadoop.fs.ChecksumFileSystem $ ChecksumFSInputChecker。< init>（ChecksumFileSystem.java:125）
at org.apache.hadoop.fs.ChecksumFileSystem.open（ChecksumFileSystem.java:283）
在org.apache.hadoop.fs.FileSystem.open（FileSystem.java:400）
在org.apache.hadoop.security.Credentials.readTokenStorageFile（Credentials.java:129）
... 5更多
11/05/16 13:39:31 WARR mapred.JobClient：读取任务输出时出错http：// jonathan-VirtualBox：50060 / tasklog？plaintext = true& tryptid = attempt_201105161222_0003_m_000003_0& filter = stdout
11/05/16 13:39:31 WARR mapred.JobClient：读取任务输出时出错http：// jonathan-VirtualBox：50060 / tasklog？plaintext = true& attemptptid = attempt_201105161222_0003_m_000003_0& filter = stderr
11/05/16 13:39:33 INFO m apred.JobClient：任务ID：attempt_201105161222_0003_m_000003_1，状态：FAILED
初始化attempt_201105161222_0003_m_000003_1错误：
java.io.IOException：异常读取文件：/ app / hadoop / tmp / mapred / local / ttprivate / taskTracker / hadoop / jobcache / job_201105161222_0003 / jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile（Credentials.java:135）
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens（TokenCache。 Java的：在org.apache.hadoop.mapred.TaskTracker.initializeJob（TaskTracker.java:1064 163）
）
在org.apache.hadoop.mapred.TaskTracker.localizeJob（TaskTracker.java:1001）
at org.apache.hadoop.mapred.TaskTracker.startNewTask（TaskTracker.java:2161）
at org.apache.hadoop.mapred.TaskTracker $ TaskLauncher.run（TaskTracker.java:2125）
导致：java.io.FileNotFoundException：文件文件：/ app / hadoop / tmp / mapred / local / ttprivate / taskTracker / hadoop / jobcache / job_201105161222_0003 / jobToken不存在。
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus（RawLocalFileSystem.java:371）
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus（FilterFileSystem.java:245）
（org.apache.hadoop.fs.ChecksumFileSystem $ ChecksumFSInputChecker。< init>（ChecksumFileSystem.java:125）
at org.apache.hadoop.fs.ChecksumFileSystem.open（ChecksumFileSystem.java:283）
在org.apache.hadoop.fs.FileSystem.open（FileSystem.java:400）
在org.apache.hadoop.security.Credentials.readTokenStorageFile（Credentials.java:129）
... 5更多
11/05/16 13:39:33警告mapred.JobClient：读取任务输出时出错http：// jonathan-VirtualBox：50060 / tasklog？plaintext = true& tryptid = attempt_201105161222_0003_m_000003_1& filter = stdout
11/05/16 13:39:33警告mapred.JobClient：读取任务输出错误http：// jonathan-VirtualBox：50060 / tasklog？plaintext = true& attemptptid = attempt_201105161222_0003_m_000003_1& filter = stderr
11/05/16 13:39:36 INFO m apred.JobClient：任务ID：attempt_201105161222_0003_m_000003_2，状态：FAILED
初始化attempt_201105161222_0003_m_000003_2错误：
java.io.IOException：异常读取文件：/ app / hadoop / tmp / mapred / local / ttprivate / taskTracker / hadoop / jobcache / job_201105161222_0003 / jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile（Credentials.java:135）
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens（TokenCache。 Java的：在org.apache.hadoop.mapred.TaskTracker.initializeJob（TaskTracker.java:1064 163）
）
在org.apache.hadoop.mapred.TaskTracker.localizeJob（TaskTracker.java:1001）
at org.apache.hadoop.mapred.TaskTracker.startNewTask（TaskTracker.java:2161）
at org.apache.hadoop.mapred.TaskTracker $ TaskLauncher.run（TaskTracker.java:2125）
导致：java.io.FileNotFoundException：文件文件：/ app / hadoop / tmp / mapred / local / ttprivate / taskTracker / hadoop / jobcache / job_201105161222_0003 / jobToken不存在。
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus（RawLocalFileSystem.java:371）
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus（FilterFileSystem.java:245）
（org.apache.hadoop.fs.ChecksumFileSystem $ ChecksumFSInputChecker。< init>（ChecksumFileSystem.java:125）
at org.apache.hadoop.fs.ChecksumFileSystem.open（ChecksumFileSystem.java:283）
在org.apache.hadoop.fs.FileSystem.open（FileSystem.java:400）
在org.apache.hadoop.security.Credentials.readTokenStorageFile（Credentials.java:129）
... 5 more
11/05/16 13:39:36 WARR mapred.JobClient：读取任务输出时出错http：// jonathan-VirtualBox：50060 / tasklog？plaintext = true& tryptid = attempt_201105161222_0003_m_000003_2& filter = stdout
11/05/16 13:39:36 WARR mapred.JobClient：读取任务输出错误http：// jonathan-VirtualBox：50060 / tasklog？plaintext = true& attemptptid = attempt_201105161222_0003_m_000003_2& filter = stderr
11/05/16 13:39:39 INFO m apred.JobClient：工作完成：job_201105161222_0003
11/05/16 13:39:39信息mapred.JobClient：计数器：4
11/05/16 13:39:39信息mapred.JobClient：Job计数器
11/05/16 13:39:39信息mapred.JobClient：SLOTS_MILLIS_MAPS = 0
11/05/16 13:39:39信息mapred.JobClient：所花费的总时间减少等待后预留插槽（ms）= 0
11/05/16 13:39:39信息mapred.JobClient：预留插槽后等待的所有地图花费的总时间（毫秒）= 0
11/05/16 13:39:39信息mapred.JobClient：SLOTS_MILLIS_REDUCES = 0
解决方案
不要创建/ app / hadoop / tmp文件夹
它将在/ tmp / hadoop下默认创建...
。
它会工作
I need some help on figuring out why my job failed. I built a single node cluster just to try it out. I followed the example here.
Everything seems to be working correctly. I formatted the namenode and am able to connect to the jobtracker, datanode, and namenode via the web interface. I am able to start and stop all the hadoop services.
However, when I try to run the wordcount example, I get this:
Error initializing attempt_201105161023_0002_m_000011_0:
java.io.IOException: Exception reading file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161023_0002/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:163)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1064)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1001)
at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2161)
at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:2125)
Caused by: java.io.FileNotFoundException: File file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161023_0002/jobToken does not exist.
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:371)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:400)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
... 5 more
I created the directory on local file system.
$ sudo mkdir /app/hadoop/tmp
$ sudo chown hadoop:hadoop /app/hadoop/tmp
I also modified core-site.xml:
<!-- In: conf/core-site.xml -->
<property>
<name>hadoop.tmp.dir</name>
<value>/app/hadoop/tmp</value>
<description>A base for other temporary directories.</description>
</property>
When I format the namenode, it creates the subdirectory on both local and HDFS successfully.
When I look at the result of the failed wordcount output, the error
message complains about an I/O error, on file
/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/
job_201105161023_0002/jobToken
I did some troubleshooting and can browse to this jobToken file on the
local file system, no problem. The content is something like HDTS MapReduce.job 201105161023_0002
So is it a permission issue? I made hadoop process user able to write to all the subdirectories and it was able to create the file. So what else can be wrong?
Some more detailed output from the command line:
hadoop@jonathan-VirtualBox:/usr/local/hadoop/hadoop-0.20.203.0$ bin/hadoop jar hadoop-examples-0.20.203.0.jar wordcount app/download app/output4
11/05/16 13:38:56 INFO input.FileInputFormat: Total input paths to process : 3
11/05/16 13:39:05 INFO mapred.JobClient: Running job: job_201105161222_0003
11/05/16 13:39:06 INFO mapred.JobClient: map 0% reduce 0%
11/05/16 13:39:17 INFO mapred.JobClient: Task Id : attempt_201105161222_0003_m_000004_0, Status : FAILED
Error initializing attempt_201105161222_0003_m_000004_0:
java.io.IOException: Exception reading file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161222_0003/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:163)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1064)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1001)
at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2161)
at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:2125)
Caused by: java.io.FileNotFoundException: File file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161222_0003/jobToken does not exist.
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:371)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:400)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
... 5 more
11/05/16 13:39:21 WARN mapred.JobClient: Error reading task outputhttp://jonathan-VirtualBox:50060/tasklog?plaintext=true&attemptid=attempt_201105161222_0003_m_000004_0&filter=stdout
11/05/16 13:39:21 WARN mapred.JobClient: Error reading task outputhttp://jonathan-VirtualBox:50060/tasklog?plaintext=true&attemptid=attempt_201105161222_0003_m_000004_0&filter=stderr
11/05/16 13:39:22 INFO mapred.JobClient: Task Id : attempt_201105161222_0003_m_000004_1, Status : FAILED
Error initializing attempt_201105161222_0003_m_000004_1:
java.io.IOException: Exception reading file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161222_0003/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:163)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1064)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1001)
at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2161)
at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:2125)
Caused by: java.io.FileNotFoundException: File file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161222_0003/jobToken does not exist.
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:371)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:400)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
... 5 more
11/05/16 13:39:22 WARN mapred.JobClient: Error reading task outputhttp://jonathan-VirtualBox:50060/tasklog?plaintext=true&attemptid=attempt_201105161222_0003_m_000004_1&filter=stdout
11/05/16 13:39:22 WARN mapred.JobClient: Error reading task outputhttp://jonathan-VirtualBox:50060/tasklog?plaintext=true&attemptid=attempt_201105161222_0003_m_000004_1&filter=stderr
11/05/16 13:39:23 INFO mapred.JobClient: Task Id : attempt_201105161222_0003_m_000004_2, Status : FAILED
Error initializing attempt_201105161222_0003_m_000004_2:
java.io.IOException: Exception reading file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161222_0003/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:163)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1064)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1001)
at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2161)
at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:2125)
Caused by: java.io.FileNotFoundException: File file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161222_0003/jobToken does not exist.
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:371)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:400)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
... 5 more
11/05/16 13:39:23 WARN mapred.JobClient: Error reading task outputhttp://jonathan-VirtualBox:50060/tasklog?plaintext=true&attemptid=attempt_201105161222_0003_m_000004_2&filter=stdout
11/05/16 13:39:23 WARN mapred.JobClient: Error reading task outputhttp://jonathan-VirtualBox:50060/tasklog?plaintext=true&attemptid=attempt_201105161222_0003_m_000004_2&filter=stderr
11/05/16 13:39:30 INFO mapred.JobClient: Task Id : attempt_201105161222_0003_m_000003_0, Status : FAILED
Error initializing attempt_201105161222_0003_m_000003_0:
java.io.IOException: Exception reading file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161222_0003/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:163)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1064)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1001)
at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2161)
at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:2125)
Caused by: java.io.FileNotFoundException: File file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161222_0003/jobToken does not exist.
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:371)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:400)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
... 5 more
11/05/16 13:39:31 WARN mapred.JobClient: Error reading task outputhttp://jonathan-VirtualBox:50060/tasklog?plaintext=true&attemptid=attempt_201105161222_0003_m_000003_0&filter=stdout
11/05/16 13:39:31 WARN mapred.JobClient: Error reading task outputhttp://jonathan-VirtualBox:50060/tasklog?plaintext=true&attemptid=attempt_201105161222_0003_m_000003_0&filter=stderr
11/05/16 13:39:33 INFO mapred.JobClient: Task Id : attempt_201105161222_0003_m_000003_1, Status : FAILED
Error initializing attempt_201105161222_0003_m_000003_1:
java.io.IOException: Exception reading file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161222_0003/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:163)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1064)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1001)
at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2161)
at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:2125)
Caused by: java.io.FileNotFoundException: File file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161222_0003/jobToken does not exist.
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:371)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:400)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
... 5 more
11/05/16 13:39:33 WARN mapred.JobClient: Error reading task outputhttp://jonathan-VirtualBox:50060/tasklog?plaintext=true&attemptid=attempt_201105161222_0003_m_000003_1&filter=stdout
11/05/16 13:39:33 WARN mapred.JobClient: Error reading task outputhttp://jonathan-VirtualBox:50060/tasklog?plaintext=true&attemptid=attempt_201105161222_0003_m_000003_1&filter=stderr
11/05/16 13:39:36 INFO mapred.JobClient: Task Id : attempt_201105161222_0003_m_000003_2, Status : FAILED
Error initializing attempt_201105161222_0003_m_000003_2:
java.io.IOException: Exception reading file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161222_0003/jobToken
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:135)
at org.apache.hadoop.mapreduce.security.TokenCache.loadTokens(TokenCache.java:163)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1064)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1001)
at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2161)
at org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:2125)
Caused by: java.io.FileNotFoundException: File file:/app/hadoop/tmp/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201105161222_0003/jobToken does not exist.
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:371)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:125)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:283)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:400)
at org.apache.hadoop.security.Credentials.readTokenStorageFile(Credentials.java:129)
... 5 more
11/05/16 13:39:36 WARN mapred.JobClient: Error reading task outputhttp://jonathan-VirtualBox:50060/tasklog?plaintext=true&attemptid=attempt_201105161222_0003_m_000003_2&filter=stdout
11/05/16 13:39:36 WARN mapred.JobClient: Error reading task outputhttp://jonathan-VirtualBox:50060/tasklog?plaintext=true&attemptid=attempt_201105161222_0003_m_000003_2&filter=stderr
11/05/16 13:39:39 INFO mapred.JobClient: Job complete: job_201105161222_0003
11/05/16 13:39:39 INFO mapred.JobClient: Counters: 4
11/05/16 13:39:39 INFO mapred.JobClient: Job Counters
11/05/16 13:39:39 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=0
11/05/16 13:39:39 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
11/05/16 13:39:39 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
11/05/16 13:39:39 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0
解决方案
Dont Create /app/hadoop/tmp folder It will create by default under /tmp/hadoop...
So remove that property from conf file . It will work
这篇关于Hadoop wordcount无法运行 - 在解码hadoop错误消息时需要帮助的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
