启动Master时出现HBase错误IllegalStateException:hsync [英] HBase Error IllegalStateException when starting Master: hsync

问题描述

我正在尝试在hadoop群集上安装HBase，无法弄清楚为什么从start-hbase.sh调用时HMaster无法启动.日志文件指示hsync存在问题.我已经确认zookeeper在分布式模式下可以正常运行，并且通过spark在使用hadoop集群方面没有任何问题.尝试启动HBase时，区域服务器将在所有数据节点上启动.

I'm trying to install HBase on a hadoop cluster and can't figure out why the HMaster fails to initiate when called from start-hbase.sh. The log files indicate an issue with hsync. I have confirmed that zookeeper is running correctly in distributed mode,and I have not had any issues working with the hadoop cluster through spark. When attempting to start HBase the region servers start on all data nodes.

我有hadoop版本3.0.0，zookeeper 3.4.11和hbase 2.0.0 beta 1.

I have hadoop version 3.0.0, zookeeper 3.4.11 and hbase 2.0.0 beta 1.

我已经清除了zookeeper hbase节点和hdfs hbase目录，并且没有运气就重新启动了.我将myid zookeeper文件存储在与Zookeeper和hbase区域服务器的.pid文件相同的目录中/usr/local/zookeeper/data/

I have cleared out the zookeeper hbase node and hdfs hbase directory and restarted with no luck. I have the myid zookeeper files stored in the same directory as the .pid files for zookeeper and hbase region servers /usr/local/zookeeper/data/

下面我摘录了与hsync异常有关的hbase主日志文件.

Below I have an excerpt of the hbase master log file pertaining to the hsync exception.

2018-02-08 17:26:54,786错误[master/MasterNode/MasterNode:16000] master.HMaster:无法成为活动的master java.lang.IllegalStateException:过程WAL依赖于在组件故障期间进行hsync进行正确操作的能力，但是基础文件系统不支持这样做.请检查"hbase.procedure.store.wal.use.hsync"的配置值以设置所需的鲁棒性级别，并确保"hbase.wal.dir"的配置值指向可以提供此文件系统的安装. 在org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1036) 在org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:374) 在org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:532) 在org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1232) 在org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1145) 在org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:837)上 在org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2026) 在org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:547) 在java.lang.Thread.run(Thread.java:748) 2018-02-08 17:26:54,787错误[master/MasterNode/MasterNode:16000] master.HMaster:主服务器中止:加载的协处理器为:[] 2018-02-08 17:26:54,787错误[master/MasterNode/MasterNode:16000] master.HMaster:未处理的异常.开始关机. java.lang.IllegalStateException:过程WAL依赖于在组件故障期间进行hsync进行正确操作的能力，但是基础文件系统不支持这样做.请检查"hbase.procedure.store.wal.use.hsync"的配置值以设置所需的鲁棒性级别，并确保"hbase.wal.dir"的配置值指向可以提供此文件系统的安装. 在org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1036) 在org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:374) 在org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:532) 在org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1232) 在org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1145) 在org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:837)上 在org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2026) 在org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:547) 在java.lang.Thread.run(Thread.java:748)

2018-02-08 17:26:54,786 ERROR [master/MasterNode/MasterNode:16000] master.HMaster: Failed to become active master java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it. at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1036) at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:374) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:532) at org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1232) at org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1145) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:837) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2026) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:547) at java.lang.Thread.run(Thread.java:748) 2018-02-08 17:26:54,787 ERROR [master/MasterNode/MasterNode:16000] master.HMaster: Master server abort: loaded coprocessors are: [] 2018-02-08 17:26:54,787 ERROR [master/MasterNode/MasterNode:16000] master.HMaster: Unhandled exception. Starting shutdown. java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it. at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1036) at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:374) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:532) at org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1232) at org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1145) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:837) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2026) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:547) at java.lang.Thread.run(Thread.java:748)

下面是hbase-site.xml文件

Below is the hbase-site.xml file

<property>
    <name>hbase.master</name>
    <value>localhost:60000</value>
    <description>The host and port that the HBase master runs at.A value of 'lo$
</property>
<property>
    <name>hbase.zookeeper.property.clientPort</name>
    <value>2181</value>
    <description>Property from ZooKeeper's config zoo.cfg.The port at which the$
</property>
<property>
    <name>hbase.zookeeper.quorum</name>
    <value>MasterNode,Slave1,Slave2</value>
    <description>Comma separated list of servers in the ZooKeeper Quorum.For ex$
</property>
<property>
    <name>hbase.rootdir</name>
    <value>hdfs://MasterNode:9000/hbase</value>
</property>
<property>
    <name>hbase.tmp.dir</name>
    <value>/usr/local/zookeeper/data</value>
</property>
<property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/usr/local/zookeeper/data</value>
</property>
<property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
</property>
<property>
    <name>zookeeper.znode.parent</name>
    <value>/hbase</value>
</property>

以下是对hbase-env.sh文件设置所做的更改

Below are the alterations made to the hbase-env.sh file settings

export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export HBASE_HOME=/usr/local/hadoop/HBase
export HADOOP_HOME=/usr/local/hadoop
export HBASE_MANAGES_ZK=false
export HBASE_PID_DIR=/usr/local/zookeeper/data

下面是core-site.xml

Below is the core-site.xml

<property>
    <name>hadoop.tmp.dir</name>
    <value>/app/hadoop/tmp</value>
</property>
<property>
    <name>fs.default.name</name>
    <value>hdfs://MasterNode:9000</value>
</property>

下面是zoo.cfg

Below is the zoo.cfg

tickTime=2000
dataDir=/usr/local/zookeeper/data
dataLogDir=/usr/local/zookeeper/log
clientPort=2181
initLimit=5
syncLimit=2
server.1=MasterNode:2888:3888
server.2=Slave1:2888:3888
server.3=Slave2:2888:3888

以下是与hadoop设置相关的.bashrc设置

Below is the .bashrc settings related to the hadoop setup

export HADOOP_HOME=/usr/local/hadoop
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_PREFIX=/usr/local/hadoop
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_CONFDIR=$HADOOP_HOME/etc/hadoop
export HADOOP_OPTS="-Djava.library.path=$HADOOP_PREFIX/lib"
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/
export PATH=$PATH:$HADOOP_HOME/bin
export JAVA_LIBRARY_PATH=$HADOOP_HOME/lib/native:$JAVA_LIBRARY_PATH
export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export HBASE_HOME=/usr/local/hadoop/HBase
export PATH=$PATH:$HBASE_HOME/bin

推荐答案

我遇到了同样的问题，我的解决方案如下: 编辑hbase-site.xml并添加以下内容:

I encountered the same problem and my solution is as follows: Edit hbase-site.xml and add the following:

<property>
  <name>hbase.unsafe.stream.capability.enforce</name>
  <value>false</value>
</property>

该属性控制HBase是否检查流功能(hflush/hsync).更多详细信息，请参见

The property controls whether HBase will check for stream capabilities (hflush/hsync).More details refer to the official website.

这篇关于启动Master时出现HBase错误IllegalStateException:hsync的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！