为什么运行hadoop时数据节点会关闭? [英] Why does data node shut down when I run hadoop?
本文介绍了为什么运行hadoop时数据节点会关闭?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我在VirtualBox的Ubuntu 11.0.4上安装了hadoop 1.0.4(与我的主机名相同),以某种方式关闭数据节点,在日志文件中出现以下错误:
/ *************** ***********************
STARTUP_MSG:启动DataNode
STARTUP_MSG:host = VirtualBox / 127.0.1.1
STARTUP_MSG: args = []
STARTUP_MSG:version = 1.0.4
STARTUP_MSG:build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290;由'hortonfo'在Wed Oct 3 05:13:58 UTC 2012编辑
****************************** ****************************** /
2013-08-18 19:52:21,301 INFO org.apache。 hadoop.metrics2.impl.MetricsConfig:从hadoop-metrics2.properties加载属性
2013-08-18 19:52:21,394 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter:MBean for source MetricsSystem,sub =统计已注册。
2013-08-18 19:52:21,412 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl:计划的10秒快照周期。
2013-08-18 19:52:21,417 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl:DataNode度量系统启动
2013-08-18 19:52:23,207 INFO org.apache。 hadoop.metrics2.impl.MetricsSourceAdapter:注册源ugi的MBean。
2013-08-18 19:52:23,276 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl:源名称ugi已经存在!
2013-08-18 19:52:26,887错误org.apache.hadoop.hdfs.server.datanode.DataNode:java.io.IOException:/ app / hadoop / tmp / dfs / data中的不兼容namespaceID:namenode namespaceID = 457871323; datanode namespaceID = 2066655210
位于org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:232)
位于org.apache.hadoop.hdfs.server.datanode.DataStorage。 recoverTransitionRead(DataStorage.java:147)
位于org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:385)
位于org.apache.hadoop.hdfs.server。 datanode.DataNode。< init>(DataNode.java:299)
at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1582)
at org.apache .hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)
2013-08-18 19:52:26,903 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:SHUTDOWN_MSG:
/ ** ************************************************** ********
SHUTDOWN_MSG:在VirtualBox关闭DataNode / 127.0.1.1
*********************** ************************************* /
任何想法为什么?我该如何解决它?
解决方案
<我已经有过几次这样的事情了。如果重新启动数据节点没有帮助,请执行以下操作:
- 重新启动Hadoop
- 转到/ app / hadoop / tmp / dfs / name / current
- 打开VERSION(例如
vim VERSION
) - 记录namespaceID
- 转至/ app / hadoop / tmp / dfs / data / current
- 打开VERSION
vim版本
) - 将namespaceID替换为您在步骤4中记录的命名空间ID。 >
这应该解决问题。
I have hadoop 1.0.4 installed on ubuntu 11.0.4 in VirtualBox(same as my hostname), somehow the data node shuts down giving the following error in the log files
/************************************************************ STARTUP_MSG: Starting DataNode STARTUP_MSG: host = VirtualBox/127.0.1.1 STARTUP_MSG: args = [] STARTUP_MSG: version = 1.0.4 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct 3 05:13:58 UTC 2012 ************************************************************/ 2013-08-18 19:52:21,301 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2013-08-18 19:52:21,394 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 2013-08-18 19:52:21,412 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2013-08-18 19:52:21,417 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started 2013-08-18 19:52:23,207 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered. 2013-08-18 19:52:23,276 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists! 2013-08-18 19:52:26,887 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs in /app/hadoop/tmp/dfs/data: namenode namespaceID = 457871323; datanode namespaceID = 2066655210 at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:232) at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:147) at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:385) at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:299) at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1582) at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521) at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539) at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665) at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682) 2013-08-18 19:52:26,903 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down DataNode at VirtualBox/127.0.1.1 ************************************************************/
Any idea why?How can I fix it?
解决方案I've had this happen a few times. If restarting the data node doesn't help, then do the following:
- Restart Hadoop
- Go to /app/hadoop/tmp/dfs/name/current
- Open VERSION (i.e. by
vim VERSION
) - Record namespaceID
- Go to /app/hadoop/tmp/dfs/data/current
- Open VERSION (i.e. by
vim VERSION
) - Replace the namespaceID with the namespaceID you recorded in step 4.
This should fix the problem.
这篇关于为什么运行hadoop时数据节点会关闭?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文