什么命令“hadoop namenode -format”会做 [英] What the command "hadoop namenode -format" will do
问题描述
我试图通过教程学习Hadoop，并尝试在我的机器上执行伪分布式模式。
我的核心站点.xml 是：
<？xml version =1.0？>
<？xml-stylesheet type =text / xslhref =configuration.xsl？>
<配置>
<属性>
<名称> fs.default.name< /名称>
< value> hdfs：// localhost：9000< / value>
< description>默认文件系统的名称。其方案和权限决定FileSystem实现的URI。
< / description>
< / property>
< / configuration>
我的 hdfs-site.xml 文件是：
<？xml version =1.0？>
<？xml-stylesheet type =text / xslhref =configuration.xsl？>
<配置>
<属性>
< name> dfs.replication< / name>
<值> 1< /值>
< description>创建
文件时，可以指定实际的复制次数。
< / description>
< / property>
< / configuration>
我的 mapred-site.xml 文件是：
<？xml version =1.0？>
<？xml-stylesheet type =text / xslhref =configuration.xsl？>
<配置>
<属性>
<名称> mapred.job.tracker< / name>
< value> localhost：9001< /值>
< description> MapReduce作业追踪器的主机和端口运行
at。
< / description>
< / property>
< / configuration>
当我运行该命令时，它运行成功，但实际上它在做什么：
hadoop-1.2.1 $ bin / hadoop namenode -format
14/11/26 12:37:16 INFO namenode.NameNode：STARTUP_MSG ：
/ ******************************************** ****************
STARTUP_MSG：启动NameNode
STARTUP_MSG：host = myhost / 127.0.0.8
STARTUP_MSG：args = [-format]
STARTUP_MSG：version = 1.2.1
STARTUP_MSG：build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152;由'mattf'编译于星期一7月22日15:23:09 PDT 2013
STARTUP_MSG：java = 1.6.0_45
******************* ************************************* /
14/11 / 26 12:37:17 INFO util.GSet：映射的计算容量BlocksMap
14/11/26 12:37:17 INFO util.GSet：VM类型= 64位
14/11/26 12:37:17 INFO util.GSet：2.0％max memory = 932118528
14/11/26 12:37:17 INFO util.GSet：capacity = 2 ^ 21 = 2097152条目
14/11 / 26 12:37:17 INFO util.GSet：recommended = 2097152，actual = 2097152
14/11/26 12:37:17 INFO namenode.FSNamesystem：fsOwner = myuser
14/11/26 12:37:17 INFO namenode.FSNamesystem：supergroup = supergroup
14/11/26 12:37:17 INFO namenode.FSNamesystem：isPermissionEnabled = true
14/11/26 12:37:17 INFO namenode.FSNamesystem：dfs.block.invalidate.limit = 100
14/11/26 12:37:17 INFO namenode.FSNamesystem：isAccessTokenEnabled = false accessKeyUpdateInterval = 0 min（s），accessTokenLifetime = 0 min（s）
14/11/26 12:37:17 INFO namenode.FSEditLo g：dfs.namenode.edits.toleration.length = 0
14/11/26 12:37:17 INFO namenode.NameNode：缓存超过10次的文件名
14/11/26 12 ：37：17 INFO common.Storage：大小为115字节的图像文件/ tmp / hadoop-myuser / dfs / name / current / fsimage在0秒内保存。
14/11/26 12:37:18 INFO namenode.FSEditLog：关闭编辑日志：position = 4，editlog = / tmp / hadoop-myuser / dfs / name / current / edits
14/11 / 26 12:37:18 INFO namenode.FSEditLog：close success：truncate to 4，editlog = / tmp / hadoop-myuser / dfs / name / current / edits
14/11/26 12:37:18 INFO common.Storage：存储目录/ tmp / hadoop-myuser / dfs / name已成功格式化。
14/11/26 12:37:18信息namenode.NameNode：SHUTDOWN_MSG：
/ ************************ ************************************
SHUTDOWN_MSG：在chaitanya-OptiPlex上关闭NameNode- 3010 / 127.0.0.8
***************************************** ******************* /
可以有人请让我知道它在内部做了什么。
我经历过这些帖子，但没有正确的解释。
如何在我的机器上实际检查这一点，以便在运行命令前后可以看到差异。我是Hadoop的新手，所以这可能是一个微不足道的问题。
Hadoop namenode -format-
Hadoop namenode目录包含fsimage和编辑文件，其中
包含有关hadoop文件系统的基本信息，例如where
data available，which user created files like that -
如果格式化namenode，那么上面的信息将从namenode目录中删除
在hdfs-site.xml中指定为dfs.namenode.name.dir -
但是你仍然有hadoop的数据，但不包含namenode元数据
I am trying to learn Hadoop by following a tutorial and trying to do pseudo-distributed mode on my machine.
My core-site.xml is:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
<description>The name of the default file system. A URI whose scheme and authority determine the FileSystem implementation.
</description>
</property>
</configuration>
My hdfs-site.xml file is:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
<description>The actual number of replications can be specified when the
file is created.
</description>
</property>
</configuration>
My mapred-site.xml file is:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
<description>The host and port that the MapReduce job tracker runs
at.
</description>
</property>
</configuration>
When I run the command it ran successfully but what it is doing actually:
hadoop-1.2.1$ bin/hadoop namenode -format
14/11/26 12:37:16 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = myhost/127.0.0.8
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 1.2.1
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013
STARTUP_MSG: java = 1.6.0_45
************************************************************/
14/11/26 12:37:17 INFO util.GSet: Computing capacity for map BlocksMap
14/11/26 12:37:17 INFO util.GSet: VM type = 64-bit
14/11/26 12:37:17 INFO util.GSet: 2.0% max memory = 932118528
14/11/26 12:37:17 INFO util.GSet: capacity = 2^21 = 2097152 entries
14/11/26 12:37:17 INFO util.GSet: recommended=2097152, actual=2097152
14/11/26 12:37:17 INFO namenode.FSNamesystem: fsOwner=myuser
14/11/26 12:37:17 INFO namenode.FSNamesystem: supergroup=supergroup
14/11/26 12:37:17 INFO namenode.FSNamesystem: isPermissionEnabled=true
14/11/26 12:37:17 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
14/11/26 12:37:17 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
14/11/26 12:37:17 INFO namenode.FSEditLog: dfs.namenode.edits.toleration.length = 0
14/11/26 12:37:17 INFO namenode.NameNode: Caching file names occuring more than 10 times
14/11/26 12:37:17 INFO common.Storage: Image file /tmp/hadoop-myuser/dfs/name/current/fsimage of size 115 bytes saved in 0 seconds.
14/11/26 12:37:18 INFO namenode.FSEditLog: closing edit log: position=4, editlog=/tmp/hadoop-myuser/dfs/name/current/edits
14/11/26 12:37:18 INFO namenode.FSEditLog: close success: truncate to 4, editlog=/tmp/hadoop-myuser/dfs/name/current/edits
14/11/26 12:37:18 INFO common.Storage: Storage directory /tmp/hadoop-myuser/dfs/name has been successfully formatted.
14/11/26 12:37:18 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at chaitanya-OptiPlex-3010/127.0.0.8
************************************************************/
Can someone please let me know what it is doing internally.
I have gone through these posts but there is no correct explanation.
What exactly is hadoop namenode formatting?
hadoop namenode is not formatting
How can I check this practically on my machine so I can see the differences before and after running the command. I am new to Hadoop so this can be a trivial question.
Hadoop namenode -format
Hadoop namenode directory contains the fsimage and edit files which holds the basic information's about hadoop file system such as where is data available, which user created files like that
If you format the namenode then the above information's are deleted from namenode directory which is specified in the hdfs-site.xml as
dfs.namenode.name.dirBut you still have the datas on the hadoop but not namenode meta data
这篇关于什么命令“hadoop namenode -format”会做的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
