Namenode格式不释放datanode磁盘空间 [英] Namenode format does not free up datanode disk space

问题描述

关闭群集 ./ stop-all.sh ，然后调用 hadoop namenode -format ，我发现datanode具有相同的磁盘空间，即空间没有被释放。

为什么会这样？

解决方案

在格式化NameNode之前，您可以手动删除DataNode上的数据 >

 用法：hadoop fs -rmr URI [URI ...] 
  

递归版本的删除。
示例：

  hadoop fs -rmr / user / hadoop / dir 
 hadoop fs -rmr hdfs： //nn.example.com/user/hadoop/dir 
  

退出代码：

成功返回0，错误返回-1。

---

或者

无论名称节点是什么，数据节点都应该重新格式化。我在这里看到两种方法：

1. 为了重新格式化集群，我们称之为start-dfs -format或者创建一个特殊的脚本格式-dfs。这会将集群组件全部格式化在一起。问题是格式化后它是否应该启动集群？


2. 仅格式化名称节点。当数据节点连接到名称节点时，如果它看到命名空间为空且其cTime = 0，它将告诉它们
   格式化它们的存储目录。
   这种方法的缺点是，如果错误地连接到空名称节点，我们可以从另一个集群中释放数据节点的块。

https://issues.apache.org/jira/browse / HDFS-107

After shutting down the cluster ./stop-all.sh, and then invoking a hadoop namenode -format, I see that the datanodes have the same disk space i.e. the space has not been freed up.

Why is that?

解决方案

You can delete manually data on DataNode before formatting NameNode

rmr

Usage: hadoop fs -rmr URI [URI …]

Recursive version of delete. Example:

hadoop fs -rmr /user/hadoop/dir
hadoop fs -rmr hdfs://nn.example.com/user/hadoop/dir

Exit Code:

Returns 0 on success and -1 on error.

---

Alternatively

Data-nodes should be reformatted whenever the name-node is. I see 2 approaches here:

1. In order to reformat the cluster we call "start-dfs -format" or make a special script "format-dfs". This would format the cluster components all together. The question is whether it should start the cluster after formatting?
2. Format the name-node only. When data-nodes connect to the name-node it will tell them to format their storage directories if it sees that the namespace is empty and its cTime=0. The drawback of this approach is that we can loose blocks of a data-node from another cluster if it connects by mistake to the empty name-node.

https://issues.apache.org/jira/browse/HDFS-107

这篇关于Namenode格式不释放datanode磁盘空间的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！