hbase-site.xml中的zookeeper quorum设置究竟是什么？ [英] What exactly is the zookeeper quorum setting in hbase-site.xml?

问题描述

hbase-site.xml中的zookeeper仲裁设置究竟是什么？ 正如 hbase-default.xml，这里是设置：

ZooKeeper Quorum中逗号分隔的服务器列表。例如，host1.mydomain.com，host2.mydomain.com，host3.mydomain.com。默认情况下，对于本地操作和伪分布式操作模式，将其设置为localhost。对于完全分布式设置，应将其设置为ZooKeeper法定服务器的完整列表。如果HBASE_MANAGES_ZK在hbase-env.sh中设置，这是我们将启动/停止ZooKeeper的服务器列表。

这是什么确实已被Edward J. Yoon回答 here。为了清楚起见，我编辑了一下：

Apache Zookeeper是分布式应用程序的协调服务，像Google的Chubby。许多项目使用zookeeper，我们（Apache Hama）也使用zookeeper来实现批量同步并行计算框架的屏障同步。

今天，我调查了更多关于paxos和动态法定人数Zookeeper项目的功能，以更好地命名类 org.apache.hama.zookeeper.QuorumPeer 。由于文档不够（ http：//hadoop.apache。 org / zookeeper / docs / r3.0.0 / api / index.html ），我不理解法定人数的含义，因为这个术语对我来说有点奇怪。但是，org.apache.hama.zookeeper.QuorumPeer是正确的名字！ xD

那么，什么是法定人数，为什么我们需要一个法定人数呢？

根据维基百科，法定人数是开展该组织业务所需的审议机构成员的最低人数。通常情况下，这是大部分预计会在那里的人，尽管许多机构可能会有更低或更高的法定人数。

- 容错机制是分布式系统的重要功能之一。 Quorum算法用于防止裂脑情况。当出现裂脑情况时，根据Quorum算法，zookeeper确定主分区和辅助分区。然后，主要组中的服务器接收并处理用户的请求，而次要组中的服务器变为只读。

这个系统何时从裂脑状态恢复？当他们再次合并到一个分区时。在内部，zookeeper使用原子广播协议而不是Paxos。

您还应该阅读原始版本，以防我错误地翻译了他试图呈现的概念。

我对 Apache Zookeeper a>是否明确定义了跨几个预定义主机的复制仲裁。如果未达到此法定数量，那么将不同意的分区拆分为辅助分区，直至Zookeeper能够将它们重新与主分区集成为止。

这为Hadoop的<一个href =http://en.wikipedia.org/wiki/Eventual_consistency =noreferrer>最终一致性模型。与此同时，HBase目前正在进一步整合Zookeeper及其代码。

What exactly is the zookeeper quorum setting in hbase-site.xml?

解决方案

As described in hbase-default.xml, here's the setting:

Comma separated list of servers in the ZooKeeper Quorum. For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com". By default this is set to localhost for local and pseudo-distributed modes of operation. For a fully-distributed setup, this should be set to a full list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in hbase-env.sh this is the list of servers which we will start/stop ZooKeeper on.

What this actually does has been answered by Edward J. Yoon here. With editing on my part, for clarity:

The Apache Zookeeper is a coordination service for distributed applications, like Google's Chubby. Many projects uses zookeeper, and we (Apache Hama) also use zookeeper for barrier synchronization of Bulk Synchronous Parallel computing framework.

Today, I surveyed more about the paxos and dynamic quorum features of the Zookeeper project, to better name the class org.apache.hama.zookeeper.QuorumPeer. Because of documentation is not enough ( http://hadoop.apache.org/zookeeper/docs/r3.0.0/api/index.html ), I didn't understand the meaning of "quorum", as this term was somewhat odd to me. But, "org.apache.hama.zookeeper.QuorumPeer" is the proper name!! xD

So, what is the Quorum and why do we need a Quorum?

According to Wikipedia, Quorum is the minimum number of members of a deliberative body necessary to conduct the business of that group. Ordinarily, this is a majority of the people expected to be there, although many bodies may have a lower or higher quorum.

As you know, a Fault-Tolerant mechanism is one of the important functions of distributed system. The Quorum algorithm is used to prevent a split-brain condition. When split-brain condition occurs, according to the Quorum algorithm, zookeeper determines the "Primary Partition" and "Secondary Partition". Then, the servers in primary group receive and process user's request, and the servers in secondary group become read-only.

When does this system recover from a split-brain condition? When they're merged to one partition again. Internally, zookeeper uses atomic broadcast protocol instead of Paxos.

You should also read the original version, in case I mistranslated the concepts he was trying to present.

My understanding of the quorum mechanism in Apache Zookeeper is it explicitly defines a replication quorum across several pre-defined hosts. If this quorum is not met, the partitions that disagree are split off to a secondary partition until Zookeeper can reintegrate them with the primary partition.

This adds more granularity to Hadoop's eventual consistency model. HBase, meanwhile, is currently in the process of further integrating Zookeeper with its code.

这篇关于hbase-site.xml中的zookeeper quorum设置究竟是什么？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！