了解Elasticsearch的write_consistency和仲裁规则 [英] Understanding the write_consistency and quorum rule of Elasticsearch

问题描述

根据elasticsearch文档，write_consistency级仲裁的规则为：

According to the elasticsearch documentation, the rule for write_consistency level quorum is:

quorum（> replicas / 2 + 1）

quorum (>replicas/2+1)

使用ES 0.19.10，在具有16个分片/ 3个副本的设置中，我们将获得
16个主分片
48个副本

Using ES 0.19.10, on a setup with 16 shards / 3 replicas we will get 16 primary shards 48 replicas

运行2个节点，我们将有16个（主节点）+ 16（副本）= 32个活动分片。

Running 2 nodes, we will have 16(primary) + 16(replicas) = 32 active shards.

要满足仲裁规则，仲裁> 48 / 2 + 1 = 25个活动分片。

For the quorum rule to be met, quorum > 48/2 + 1 = 25 active shards.

现在，测试证明不然，直到我们有3个节点在运行时，write_consistency级别才达到（写操作超时）。这种方式很有意义，因为在此设置中我们可以在每组2个节点之间获得一个裂脑，但是我不太了解该规则应该如何工作？我在这里使用错误的数字吗？

Now, testing this proves otherwise, write_consistency level is not met (write operations times out) until we have 3 nodes running. This kind of makes sense, since we could get a split-brain between groups of 2 nodes each in this setup, but I dont quite understand how this rule is supposed to work? Am I using the wrong numbers here?

推荐答案

主要分片计数实际上并不重要，因此我将替换它

Primary shard count doesn't actually matter, so I'm going to replace it with N.

如果您有一个包含N个分片和2个副本的索引，则复制组中将有3个分片。这意味着法定人数为两个：主副本加上一个副本。您需要两个活动的分片（通常意味着两个活动的计算机）来满足写入一致性参数

If you have an index with N shards and 2 replicas, there are three shards in the replication group. This means the quorum is two: the primary plus one of the replicas. You need two active shards, which usually means two active machines, to satisfy the write consistency parameter

具有N个分片和3个副本的索引在复制组中具有四个分片（主要副本+ 3个副本），因此法定人数为3。

An index with N shards and 3 replicas has four shards in the replication group (primary + 3 replicas), so a quorum is three.

具有N个分片和1个副本的索引是一种特殊情况，因为您无法真正拥有只有两个碎片。仅使用一个副本，Elasticsearch仅需要一个活动分片（例如主分区），因此 quorum 设置与 one

An index with N shards and 1 replica is a special case, since you can't really have a quorum with only two shards. With only one replica, Elasticsearch only requires a single active shard (e.g. the primary), so the quorum setting is identical to the one setting for this particular arrangement.

一些注意事项：

• 0.19确实很旧，您绝对应该绝对升级。自该版本发布以来，我什至无法计数已添加了多少错误修正和性能改进：）

• 0.19 is really old, you should definitely, absolutely, positively upgrade. I can't even count how many bugfixes and performance improvements have been added since that release :)

写一致性仅是网关检查。在执行索引请求之前，节点将进行一次稻草轮询以查看是否满足write_consistency。如果是，它将尝试执行索引并推送复制。这并不保证副本会成功...它们很容易失败，您会在响应中看到它。

Write consistency is merely a gateway check. Before executing the indexing request, the node will do a straw-poll to see if write_consistency is met. If it is, it tries to execute the index and push the replication. This doesn't guarantee that the replicas will succeed...they could easily fail and you'll see it in the response. It is simply a mechanism to halt the indexing process if the consistency setting is not satisfied.

具有两个节点的完全复制设置是1个主分片+ 1个副本。每个节点都有完整的数据集。没有理由拥有更多副本，因为ES拒绝将相同数据的副本放在同一台计算机上（没有意义，对HA毫无帮助）。无法索引只是写一致性的副作用，但它指出了设置中的一个更大问题：）

A "fully replicated" setup with two nodes is 1 primary shard + 1 replica. Each node has a complete set of data. There is no reason to have more replicas, since ES refuses to put the copies of the same data on the same machine (doesn't make sense, doesn't help HA). The inability to index is just a side effect of write consistency, but it's pointing out a bigger problem with your setup :)

这篇关于了解Elasticsearch的write_consistency和仲裁规则的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！