Hadoop HA.配置了自动故障转移，但是直到再次启动NN以后，备用NN才处于活动状态 [英] Hadoop HA. Auto failover configured but Standby NN doesn't become active until NN is started again

问题描述

我正在使用Hadoop 2.6.0-cdh5.6.0.我已经配置了HA.我正在显示活动(NN1)和备用namenodes(NN2).现在，当我向活动的namenode(NN1)发出终止信号时，备用的namenode(NN2)才变为活动状态，直到我重新启动NN1为​​止.再次启动NN1之后，它进入待机状态，而NN2进入活动状态.我尚未配置"ha.zookeeper.session-timeout.ms"参数，因此我假设它将默认为5秒.在等待活动和备用NN之前，我正在等待时间完成.

I am using Hadoop 2.6.0-cdh5.6.0. I have configured HA. I have active(NN1) and standby namenodes(NN2) being displayed. Now when i issue a kill signal to the active namenode(NN1) the standby namenode(NN2) does not become active until I start the NN1 back again. After starting the NN1 again it takes the standby state and NN2 takes the active state. I haven't configured the "ha.zookeeper.session-timeout.ms" parameter, so I am assuming it would be default to 5 seconds. I am waiting for the time to complete before I check for the Active and Standby NNs.

我的core-site.xml

My core-site.xml

<configuration>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://mycluster/</value>
  </property>
  <property>
    <name>hadoop.proxyuser.mapred.groups</name>
    <value>*</value>
  </property>
  <property>
    <name>hadoop.proxyuser.mapred.hosts</name>
    <value>*</value>
  </property>
  <property>
    <name>ha.zookeeper.quorum</name>
    <value>172.17.5.107:2181,172.17.3.88:2181,172.17.5.128:2181</value>
  </property>
</configuration>

我的hdfs-site.xml

My hdfs-site.xml

<configuration>
  <property>
   <name>dfs.permissions.superusergroup</name>
   <value>hadoop</value>
  </property>
  <property>
   <name>dfs.namenode.name.dir</name>
   <value>file:///data/1/dfs/nn</value>
  </property>
  <property>
   <name>dfs.datanode.data.dir</name>
   <value>file:///data/1/dfs/dn</value>
  </property>
  <property>
    <name>dfs.nameservices</name>
    <value>mycluster</value>
  </property>
  <property>
    <name>dfs.ha.namenodes.mycluster</name>
    <value>nn1,nn2</value>
  </property>
  <property>
    <name>dfs.namenode.rpc-address.mycluster.nn1</name>
    <value>172.17.5.107:8020</value>
  </property>
  <property>
    <name>dfs.namenode.rpc-address.mycluster.nn2</name>
    <value>172.17.3.88:8020</value>
  </property>
  <property>
    <name>dfs.namenode.http-address.mycluster.nn1</name>
    <value>172.17.5.107:50070</value>
  </property>
  <property>
    <name>dfs.namenode.http-address.mycluster.nn2</name>
    <value>172.17.3.88:50070</value>
  </property>
  <property>
    <name>dfs.namenode.shared.edits.dir</name>
    <value>qjournal://172.17.5.107:8485;172.17.3.88:8485;172.17.5.128:8485/mycluster</value>
  </property>
  <property>
    <name>dfs.client.failover.proxy.provider.mycluster</name>
    <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
  </property>
  <property>
    <name>dfs.ha.fencing.methods</name>
    <value>sshfence</value>
  </property>
  <property>
    <name>dfs.ha.fencing.ssh.private-key-files</name>
    <value>/root/.ssh/id_rsa</value>
  </property>
  <property>
    <name>dfs.ha.automatic-failover.enabled</name>
    <value>true</value>
  </property>
  <property>
    <name>dfs.journalnode.edits.dir</name>
    <value>/data/1/dfs/jn</value>
  </property>
</configuration>

我的zoo.cfg

maxClientCnxns=50
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/var/lib/zookeeper
# the port at which the clients will connect
clientPort=2181
# the directory where the transaction logs are stored.
dataLogDir=/var/lib/zookeeper

推荐答案

sshfence存在问题.向hdfs用户授予权限或将其更改为root用户

There was a problem with the sshfence. Grant the permissions to hdfs user or change it to root user

  <property>                                                                                   
    <name>dfs.client.failover.proxy.provider.mycluster</name>                                  
    <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>   
  </property>                                                                                  
  <property>                                                                                   
    <name>dfs.ha.fencing.methods</name>                                                        
    <value>sshfence(root)</value>                                                              
  </property>                                                                                  
  <property>                                                                                   
    <name>dfs.ha.fencing.ssh.private-key-files</name>                                          
    <value>/var/lib/hadoop-hdfs/.ssh/id_rsa</value>                                            
  </property>                                                                                  
  <property>                                                                                   
    <name>dfs.ha.automatic-failover.enabled</name>                                             
    <value>true</value>                                                                        
  </property>                                                                                  
  <property>                                                                                   
    <name>dfs.journalnode.edits.dir</name>                                                     
    <value>/data/1/dfs/jn</value>                                                              
  </property>                                                                                  
</configuration>                                                                               

这篇关于Hadoop HA.配置了自动故障转移，但是直到再次启动NN以后，备用NN才处于活动状态的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！