Stand-by-namenode是否也在做secondary-namenode的工作？ [英] Is Stand-by-namenode doing the job of Secondary-namenode also?

问题描述

朋友们，我知道在hadoop2中，当我们配置高可用性时，不需要配置辅助名称节点/检查点节点/备份节点。通过一种新的机制，可用性由活动和备用名称节点之间共享的编辑给出。

我的问题是，辅助名称节点功能是定期将编辑文件与fsimage文件合并，因此在hadoop1世界中提供了2个好处1）限制了编辑的大小文件和2）通过保持fsimage接近最新来减少重启时间。

因此，如果启用高可用性并且不需要辅助名称节点。那么谁来做fsimage的编辑呢？或者是现在由于一些架构/流程变化而不需要的步骤。

帮助我理解它。

解决方案

有两种部署HDFS HA的方式（注意这是当前的2.7.1状态，如果您在2016年某个时候登陆此帖子，事情可能发生了变化）：

• 共享NFS，其中活动和备用NameNode实际上处理相同的文件（映像和日志）。请参阅使用NFS的 HDFS高可用性。<仲裁日志管理器，其中主动和被动NameNode都依赖于一项新服务，一组至少3个JournalNode为日志编辑提供法定人数。请参阅使用仲裁日记管理器的HDFS高可用性。
  
  
  

  对于这两种配置，文档都会明确地提出问题的答案：

>






请注意，在HA群集中，备用NameNode还执行名称空间状态的检查点，因此不需要运行Secondary NameNode， HA簇中的CheckpointNode或BackupNode。事实上，这样做会是一个错误。这也允许一个正在重新配置未启用HA的HDFS群集启用HA的硬件重新使用他们之前专用于Secondary NameNode的硬件。







Friends, I came to know that in hadoop2 when we configure high availability there is no need to configure a secondary-name-node/checkpoint-node/backup-node. With a new kind of mechanism the availability is given by edits shared among the active and standby namenodes.

My question is, secondary-name-node functionality is to merge the edits file with fsimage file periodically, thus gives 2 benefits in hadoop1 world 1) limits the size of edits file and 2) reduces the time of restart by keeping the fsimage nearly up to date.

Therefore, if High Availability is enabled and if secondary-name-node is not required. Then who will do the stiching of edits with fsimage? or is that step not required now due to some architectural/process changes.

Help me to understand it.

解决方案

There are two modes of deploying HDFS HA (N.B. this is the current 2.7.1 state, if you land on this post sometime post 2016 things may had changed):

• shared NFS, where the Active and Standby NameNode are actually working on the same files (image and log). See HDFS HighAvailability using NFS.

• Quorum Journal Manager, where the active and passive NameNode both rely on a new service, a set of minimum 3 JournalNodes that provide a quorum for log edits. See HDFS High Availability Using the Quorum Journal Manager.

For both of these configurations, the documentation explicitly calls out the answer to your question:

Note that, in an HA cluster, the Standby NameNode also performs checkpoints of the namespace state, and thus it is not necessary to run a Secondary NameNode, CheckpointNode, or BackupNode in an HA cluster. In fact, to do so would be an error. This also allows one who is reconfiguring a non-HA-enabled HDFS cluster to be HA-enabled to reuse the hardware which they had previously dedicated to the Secondary NameNode.

这篇关于Stand-by-namenode是否也在做secondary-namenode的工作？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！