名称节点存储什么？ [英] Name Node stores what?

问题描述

1. 对于名称节点，
   存储在主内存中的内容以及存储在辅助内存（硬盘）中的内容。




2. 我们所说的文件到块映射是什么？

   
   

解决方案

在名称节点的情况下，存储在主内存中的内容以及在辅助内存（硬盘）中存储的
。

文件到块映射，数据节点上的块位置，活动数据节点，一堆其他元数据都存储在NameNode上的内存中。当你检查NameNode状态网站时，几乎所有这些信息都存储在内存中。

存储在磁盘上的唯一东西是fsimage，编辑日志和状态日志。有趣的是，NameNode从来没有真正在磁盘上使用这些文件，除了启动时。 fsimage 和编辑文件几乎只存在于能够将NameNode备份，如果它需要停止或者崩溃。

我们的意思是文件到块映射？

将文件放入HDFS时，会将其分割为多个块（可配置大小）。假设您有一个名为file.txt的文件，其大小为201MB，您的块大小为64MB。您将得到三个64MB块和一个9MB块（64 + 64 + 64 + 9 = 201）。 NameNode跟踪HDFS中的file.txt映射到这四个块的事实。 DataNodes存储块，而不是文件，因此映射对于理解数据的位置和数据是很重要的。

fsimage究竟是什么并编辑日志？

NameNode内存的最近检查点存储在 fsimage 。可以从该文件中恢复该检查点的NameNode状态（即文件 - >块映射，文件属性等）。

编辑文件是 fsimage fsimage 之后的最新更改。 NameNode出现的方式是将 fsimage 实现到内存中，然后按照它看到的顺序应用编辑他们在编辑文件中。

fsimage 和 edits 以他们的方式存在，因为每次执行HDFS操作时编辑潜在的巨大 fsimage 文件可能会很难系统。相反，编辑文件只是附加到。但是，对于NameNode启动和数据存储的原因，将编辑滚动到fsimage中是一件好事。

SecondaryNameNode是定期将 fsimage 和 edits 文件合并到一个新的检查点 fsimage 文件。这是防止编辑变得巨大的重要过程。

1. In case of "Name Node", what gets stored in main memory and what gets stored in secondary memory ( hard disk ).

2. What we mean by "file to block mapping" ?

3. What exactly is fsimage and edit logs ?

解决方案

In case of "Name Node", what gets stored in main memory and what gets stored in secondary memory ( hard disk ).

The file to block mapping, locations of blocks on data nodes, active data nodes, a bunch of other metadata is all stored in memory on the NameNode. When you check the NameNode status website, pretty much all of that information is stored in memory somewhere.

The only thing stored on disk is the fsimage, edit log, and status logs. It's interesting to note that the NameNode never really uses these files on disk, except for when it starts. The fsimage and edits file pretty much only exist to be able to bring the NameNode back up if it needs to be stopped or it crashes.

What we mean by "file to block mapping" ?

When a file is put into HDFS, it is split into blocks (of configurable size). Let's say you have a file called "file.txt" that is 201MB and your block size is 64MB. You will end up with three 64MB blocks and a 9MB block (64+64+64+9 = 201). The NameNode keeps track of the fact that "file.txt" in HDFS maps to these four blocks. DataNodes store blocks, not files, so the mapping is important to understanding where your data is and what your data is.

What exactly is fsimage and edit logs ?

A recent checkpoint of the memory of the NameNode is stored in the fsimage. The NameNode's state (i.e. file->block mapping, file properties, etc.) from that checkpoint can be restored from this file.

The edits file are all the new updates from the fsimage since the last checkpoint. These are things like a file being deleted or added. This is important for if your NameNode goes down, as it has the most recent changes since the last checkpoint stored in fsimage. The way the NameNode comes up is it materializes the fsimage into memory, and then applies the edits in the order it sees them in the edits file.

fsimage and edits exist the way they do because editing the potentially massive fsimage file every time a HDFS operation is done can be hard on the system. Instead, the edits file is simply appended to. However, for the NameNode starting up and for data storage reasons, rolling the edits into the fsimage every now and then is a good thing.

The SecondaryNameNode is the process that periodically takes the fsimage and edits file and merges them together, into a new checkpointed fsimage file. This is an important process to prevent edits from getting huge.

这篇关于名称节点存储什么？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！