取消确定Elasticsearch的write_consistency和quorum规则 [英] Undestanding the write_consistency and quorum rule of Elasticsearch

问题描述

根据弹性搜索文档，write_consistency级法定的规则是：

quorum（> replicas / 2 + 1）

使用ES 0.19.10，在16个分片/ 3个副本的设置上，我们将获得
16个主分片
48个副本

运行2个节点，我们将有16个（主）+16个（副本）= 32个活动分片。

要满足法定人数规则， 2 + 1 = 25个有效碎片。

现在，测试这样证明了，否则write_consistency级别不满足（写操作超时），直到我们有3个节点运行。这是有道理的，因为我们可以在这个设置中的每个2个节点之间得到一个分裂的大脑，但我不太明白这个规则应该是如何工作的？我在这里使用错误的数字？

解决方案

主分片计数并不重要，所以我要替换它与N.

如果您有一个索引为N个碎片和2个副本，则在复制组中有三个碎片。这意味着法定人数是两个：主要加一个副本。您需要两个活动分片，通常意味着两个活动的机器，以满足写入一致性参数

具有N个分片和3个副本的索引在复制组中有四个分片（主要+ 3个副本），所以法定人数是三个。

一个索引，N个碎片和1个副本是一个特殊情况，因为你不能真正拥有一个法定人数只有两个碎片。只有一个副本，Elasticsearch只需要一个活动的分片（例如主要的），所以 quorum 设置与一个

一些注释：

• 0.19真的很老，绝对肯定地升级。我甚至不能指望自该版本以来添加了多少错误修复和性能改进：）




• 写入一致性只是一个网关检查。在执行索引请求之前，节点将进行一个草稿轮询，以查看是否满足write_consistency。如果是，则尝试执行索引并推送复制。这不能保证副本成功，他们很容易失败，你会在回应中看到它。如果一致性设置不满足，它只是一种停止索引过程的机制。




• 具有两个节点的完全复制设置是1个主分片+ 1副本。每个节点都有一套完整的数据。没有理由有更多的副本，因为ES拒绝将同一数据的副本放在同一台机器上（没有意义，不能帮助HA）。无法索引只是写入一致性的副作用，但它指出您的设置更大的问题：）

According to the elasticsearch documentation, the rule for write_consistency level quorum is:

quorum (>replicas/2+1)

Using ES 0.19.10, on a setup with 16 shards / 3 replicas we will get 16 primary shards 48 replicas

Running 2 nodes, we will have 16(primary) + 16(replicas) = 32 active shards.

For the quorum rule to be met, quorum > 48/2 + 1 = 25 active shards.

Now, testing this proves otherwise, write_consistency level is not met (write operations times out) until we have 3 nodes running. This kind of makes sense, since we could get a split-brain between groups of 2 nodes each in this setup, but I dont quite understand how this rule is supposed to work? Am I using the wrong numbers here?

解决方案

Primary shard count doesn't actually matter, so I'm going to replace it with N.

If you have an index with N shards and 2 replicas, there are three shards in the replication group. This means the quorum is two: the primary plus one of the replicas. You need two active shards, which usually means two active machines, to satisfy the write consistency parameter

An index with N shards and 3 replicas has four shards in the replication group (primary + 3 replicas), so a quorum is three.

An index with N shards and 1 replica is a special case, since you can't really have a quorum with only two shards. With only one replica, Elasticsearch only requires a single active shard (e.g. the primary), so the quorum setting is identical to the one setting for this particular arrangement.

A few notes:

• 0.19 is really old, you should definitely, absolutely, positively upgrade. I can't even count how many bugfixes and performance improvements have been added since that release :)

• Write consistency is merely a gateway check. Before executing the indexing request, the node will do a straw-poll to see if write_consistency is met. If it is, it tries to execute the index and push the replication. This doesn't guarantee that the replicas will succeed...they could easily fail and you'll see it in the response. It is simply a mechanism to halt the indexing process if the consistency setting is not satisfied.

• A "fully replicated" setup with two nodes is 1 primary shard + 1 replica. Each node has a complete set of data. There is no reason to have more replicas, since ES refuses to put the copies of the same data on the same machine (doesn't make sense, doesn't help HA). The inability to index is just a side effect of write consistency, but it's pointing out a bigger problem with your setup :)

这篇关于取消确定Elasticsearch的write_consistency和quorum规则的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！