Elasticsearch 7.2.0：主节点尚未发现或选举，选举至少需要X个节点 [英] Elasticsearch 7.2.0: master not discovered or elected yet, an election requires at least X nodes

问题描述

我正在尝试自动化kubernetes集群中Elasticsearch节点的水平放大和缩小的过程。

I'm trying to automate the process of horizontal scale up and scale down of elasticsearch nodes in kubernetes cluster.

最初，我部署了一个Elasticsearch集群（3个主节点） ，Kubernetes集群上的3个数据和3个接收节点）。其中， cluster.initial_master_nodes 是：

Initially, I deployed an elasticsearch cluster (3 master, 3 data & 3 ingest nodes) on a Kubernetes cluster. Where, cluster.initial_master_nodes was:

cluster.initial_master_nodes:
  - master-a
  - master-b
  - master-c

然后，我执行了按比例缩小操作，将主节点3的数量减少为1（意料之中，但出于测试目的）。这样做时，我删除了 master-c ， master-b 节点并重新启动 master-具有以下设置的节点：

Then, I performed scale down operation, reduced the number of master node 3 to 1 (unexpected, but for testing purpose). While doing this, I deleted master-c, master-b nodes and restarted master-a node with the following setting:

cluster.initial_master_nodes:
  - master-a

由于elasticsearch节点（即吊舱）在重启后使用持久性卷节点， master-a 减慢了以下日志的速度：

Since the elasticsearch nodes (i.e. pods) use persistant volume, after restarting the node, the master-a slowing the following logs:

"message": "master not discovered or elected yet, an election requires at least 2 nodes with ids from [TxdOAdryQ8GAeirXQHQL-g, VmtilfRIT6KDVv1R6MHGlw, KAJclUD2SM6rt9PxCGACSA], have discovered [] which is not a quorum; discovery will continue using [] from hosts providers and [{master-a}{VmtilfRIT6KDVv1R6MHGlw}{g29haPBLRha89dZJmclkrg}{10.244.0.95}{10.244.0.95:9300}{ml.machine_memory=12447109120, xpack.installed=true, ml.max_open_jobs=20}] from last-known cluster state; node term 5, last-accepted version 40 in term 5"  }

似乎正在尝试找到 master-b 和 master-c 。

问题：

• 如何覆盖群集设置，以使 master-a 不会搜索这些已删除的节点？

• How to overwrite cluster settings so that master-a won't search for these deleted nodes?

推荐答案

cluster.initial_master_nodes 设置仅在群集首次启动时有效，但要避免在一些非常罕见的极端情况下，一旦设置好它就永远不要更改它的值，通常应该尽快将其从配置文件中删除。从参考手册中有关 cluster.initial_master_nodes ：

The cluster.initial_master_nodes setting only has an effect the first time the cluster starts up, but to avoid some very rare corner cases you should never change its value once you've set it and generally you should remove it from the config file as soon as possible. From the reference manual regarding cluster.initial_master_nodes:

重新启动集群或添加集群时不应使用此设置

You should not use this setting when restarting a cluster or adding a new node to an existing cluster.

除此之外，Elasticsearch使用基于仲裁的选举协议并说明以下内容：

Aside from that, Elasticsearch uses a quorum-based election protocol and says the following:

为确保群集仍然可用，您不得同时停止表决配置中一半或更多的节点。

您已经同时停止了三个主节点中的两个，这是其中一半以上，因此可以预期

You have stopped two of your three master-eligible nodes at the same time, which is more than half of them, so it's expected that the cluster no longer works.

参考手册还包含用于删除符合主机资格的节点的说明，您没有遵循：

The reference manual also contains instructions for removing master-eligible nodes which you have not followed:

只要集群中至少有三个主节点，通常最好一次删除一个节点，从而为群集留出足够的时间来自动调整投票配置并使容错级别适应新的节点集。

As long as there are at least three master-eligible nodes in the cluster, as a general rule it is best to remove nodes one-at-a-time, allowing enough time for the cluster to automatically adjust the voting configuration and adapt the fault tolerance level to the new set of nodes.

如果只有两个符合主机要求的节点剩下的节点将无法安全地删除，因为要求两个节点都可靠地取得进展。要删除这些节点之一，您必须首先通知Elasticsearch它不应成为投票配置的一部分，而应该将投票权赋予另一个节点。

If there are only two master-eligible nodes remaining then neither node can be safely removed since both are required to reliably make progress. To remove one of these nodes you must first inform Elasticsearch that it should not be part of the voting configuration, and that the voting power should instead be given to the other node.

接下来将描述如何在缩小到以下比例时使用 POST / _cluster / voting_config_exclusions / node_name 从投票配置中安全删除不需要的节点一个节点。

It goes on to describe how to safely remove the unwanted nodes from the voting configuration using POST /_cluster/voting_config_exclusions/node_name when scaling down to a single node.

这篇关于Elasticsearch 7.2.0：主节点尚未发现或选举，选举至少需要X个节点的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！