Hadoop TaskTracker问题 [英] Hadoop tasktracker issue
本文介绍了Hadoop TaskTracker问题的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
这是一个奇怪的问题.当我的便携式计算机断电并自行关闭时，我正在运行MR作业(单节点).由于ubuntu 10.04中缺少没有电池"警告
This is a weird problem. I was running a MR job (single node) when my lappy lost power and shutdown itself. Thanks to the lack of "no battery" warning in ubuntu 10.04
当我再次启动并尝试启动hadoop(单节点)时，一切顺利，当我从命令行发布"jps"时，将列出所有主进程和从属进程.我还尝试发布"netstat -plten | grep java".
When I started up again and tried to start hadoop (single node), everything goes well and when I issue "jps" from the command line, all the master and slave processes are listed. I also tried issuing "netstat -plten | grep java".
命令行输出:
hduser@fox:~$ jps
10476 DataNode
1819
10696 JobTracker
10813 TaskTracker
11741 Jps
10605 SecondaryNameNode
10360 NameNode
hduser@fox:~$ sudo netstat -plten | grep java
tcp 0 0 0.0.0.0:55376 0.0.0.0:* LISTEN 1000 141925 10476/java
tcp 0 0 0.0.0.0:42352 0.0.0.0:* LISTEN 1000 141790 10360/java
tcp 0 0 0.0.0.0:50070 0.0.0.0:* LISTEN 1000 142727 10360/java
tcp 0 0 0.0.0.0:50010 0.0.0.0:* LISTEN 1000 142818 10476/java
tcp 0 0 0.0.0.0:50075 0.0.0.0:* LISTEN 1000 143308 10476/java
tcp 0 0 0.0.0.0:50020 0.0.0.0:* LISTEN 1000 146358 10476/java
tcp 0 0 127.0.0.1:38533 0.0.0.0:* LISTEN 1000 143217 10813/java
tcp 0 0 102.173.156.27:54310 0.0.0.0:* LISTEN 1000 141913 10360/java
tcp 0 0 102.173.156.27:54311 0.0.0.0:* LISTEN 1000 142359 10696/java
tcp 0 0 0.0.0.0:50090 0.0.0.0:* LISTEN 1000 142827 10605/java
tcp 0 0 0.0.0.0:56779 0.0.0.0:* LISTEN 1000 142191 10605/java
tcp 0 0 0.0.0.0:55373 0.0.0.0:* LISTEN 1000 142251 10696/java
tcp 0 0 0.0.0.0:50030 0.0.0.0:* LISTEN 1000 142921 10696/java
我检查了作业跟踪器日志文件:
I checked the job tracker log file:
2013-01-09 16:40:52,286 WARN org.apache.hadoop.hdfs.DFSClient: Error Recovery for block null bad datanode[0] nodes == null
2013-01-09 16:40:52,286 WARN org.apache.hadoop.hdfs.DFSClient: Could not get block locations. Source file "/home/hduser/hadoop/tmpnew/mapred/system/jobtracker.info" - Aborting...
2013-01-09 16:40:52,286 WARN org.apache.hadoop.mapred.JobTracker: Writing to file hdfs://master:54310/home/hduser/hadoop/tmpnew/mapred/system/jobtracker.info failed!
2013-01-09 16:40:52,287 WARN org.apache.hadoop.mapred.JobTracker: FileSystem is not ready yet!
2013-01-09 16:40:52,290 WARN org.apache.hadoop.mapred.JobTracker: Failed to initialize recovery manager.
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /home/hduser/hadoop/tmpnew/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
at org.apache.hadoop.ipc.Client.call(Client.java:1070)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at $Proxy5.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at $Proxy5.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3510)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3373)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2589)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2829)
2013-01-09 16:41:02,291 WARN org.apache.hadoop.mapred.JobTracker: Retrying...
2013-01-09 16:41:02,331 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /home/hduser/hadoop/tmpnew/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
at org.apache.hadoop.ipc.Client.call(Client.java:1070)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at $Proxy5.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at $Proxy5.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3510)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3373)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2589)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2829) 我无法使用URL查看任务跟踪程序-master:50060
I am not able to see the tasktracker using the URL - master:50060
因此，我有一种感觉，任务跟踪器无法正确启动.
Therefore I have a feeling that the tasktracker is not started up correctly.
帮助表示赞赏.
推荐答案
namenode格式仍然存在错误.必须使用全新安装来设置其他用户帐户.问题解决了.
Error persisted from namenode format. Had to setup different user account with fresh installation. Problem solved.
这篇关于Hadoop TaskTracker问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
