在某些代理关闭后，如何更改主题领导者或删除分区? [英] How change topic leader or remove partition after some broker down?

问题描述

我们有带有4个代理的kafka集群和一些具有复制因子1和10分区的主题.在4个时刻中有2个时刻我们的带有kafka集群的服务器出现故障.因此，现在我们有2个具有相同主题的经纪人.当我 m运行命令时./kafka_topics.sh --zookeeper本地主机:2181-描述我我明白了:

We have kafka cluster with 4 brokers and some topics with replica factor 1 and 10 partitions. At one moment 2 of 4 our servers with kafka cluster - fail. So now we have 2 brokers with same topics. When im run command ./kafka_topics.sh --zookeeper localhost:2181 --describe im get this:

Topic:outcoming-notification-error-topic        PartitionCount:10       ReplicationFactor:1     Configs:
    Topic: outcoming-error-topic       Partition: 0    Leader: 2       Replicas: 2     Isr: 2
    Topic: outcoming-error-topic       Partition: 1    Leader: 3       Replicas: 3     Isr: 3
    Topic: outcoming-error-topic       Partition: 2    Leader: 4       Replicas: 4     Isr: 4
    Topic: outcoming-error-topic       Partition: 3    Leader: 1       Replicas: 1     Isr: 1
    Topic: outcoming-error-topic       Partition: 4    Leader: 2       Replicas: 2     Isr: 2
    Topic: outcoming-error-topic       Partition: 5    Leader: 3       Replicas: 3     Isr: 3
    Topic: outcoming-error-topic       Partition: 6    Leader: 4       Replicas: 4     Isr: 4
    Topic: outcoming-error-topic       Partition: 7    Leader: 1       Replicas: 1     Isr: 1
    Topic: outcoming-error-topic       Partition: 8    Leader: 2       Replicas: 2     Isr: 2
    Topic: outcoming-error-topic       Partition: 9    Leader: 3       Replicas: 3     Isr: 3

如何删除Leader 2 ... 4?还是我需要为此领导者删除分区，但是如何?

How can i delete Leader 2...4 ? or may be i need delete partition for this Leader , but how ?

UPD ..

我们还使用kafka_exporter来监视带有prometheus的kafka.在kafka_exporter的日志中有2个经纪人记录下来之后，我们得到以下错误:

Also we use kafka_exporter for monitoring kafka with prometheus. After 2 brokers down in log of kafka_exporter we get this errors:

level=error msg="Cannot get oldest offset of topic outcoming-error-topic partition  10: kafka server: In the middle of a leadership election, there is currently no leader for this partition and hence it is unavailable for writes." source="kafka_exporter.go:296"

推荐答案

您可以使用Kafka的 kafka-reassign-partitions.sh 进行此操作.您有两种方法，一种是生成新任务的提案，另一种是

You could use Kafka's kafka-reassign-partitions.sh to do that. You have two ways, one is generating a proposal of new assignments, and the other one is manually specifying the leaders for specific partitions.

在 kafka文档中指定的第一种方法遵循以下逻辑:

The first method, as specified on the kafka docs, follows this logic:

1.1 生成建议的分区重新分配配置

首先，您应该创建一个json文件，例如链接中提供的.让我们将其命名为 topics.json .

First, you should create a json file such as the provided in the link. Let's name it topics.json.

{
  "topics": [{"topic": "foo1"},
            {"topic": "foo2"}],
  "version":1
}

这将告诉kafka您愿意从中分配分区的主题是什么.在示例中，他希望Kafka为主题 foo1 和 foo2 提出建议.

This will tell kafka what are the topics you are willing to rellocate their partitions from. In the example, he wants Kafka to make a proposal for topics foo1 and foo2.

使用该json，调用该工具并在命令中设置活动代理列表:

With that json, call the tool and set the active broker list in the command:

kafka-reassign-partitions.sh --zookeeper $ZK_HOSTS 
--topics-to-move-json-file topics.json --broker-list "1,2,3,4,5" --generate

这将输出Kafka的投标，您可以将其保存到另一个.json文件中.例如:

This will output Kafka's proposal, which you can save into another .json file. For example:

{
  "version":1,
  "partitions":[{"topic":"foo1","partition":2,"replicas":[5,6]},
              {"topic":"foo1","partition":0,"replicas":[5,6]},
              {"topic":"foo2","partition":2,"replicas":[5,6]},
              {"topic":"foo2","partition":0,"replicas":[5,6]},
              {"topic":"foo1","partition":1,"replicas":[5,6]},
              {"topic":"foo2","partition":1,"replicas":[5,6]}]
}

如果愿意，您可以手动修改某些分配(或认为这是适当的想法，因为该工具并不完美).将json保存到文件中，例如 reassign-example.json ，该文件将在下一步中使用.

You can manually modify some of the assignments, if you want to (or think it's the proper think to do, as the tool is not perfect). Save the json into a file, for example, reassign-example.json, which will be used in the next step.

1.2.执行建议的分区重新分配

让Kafka执行提案并移动分区.为此，执行:

Let's make Kafka execute the proposal and move the partitions. For that, execute:

bin/kafka-reassign-partitions.sh --zookeeper $ZK_HOSTS 
 --reassignment-json-file reassign-example.json --execute

这将执行在 reassign-example.json 文件上定义的分区移动.

This will execute the partition movement defined on the reassign-example.json file.

第二种方法相当简单，但是您必须手动标识要重新分配的分区.例如，如果您希望将主题XXX的分区1移至代理5和6，则可以创建一个json文件( manual-reassign.json )，例如:

The second method is fairly easier, but you must manually identify the partitions you want to reassign. For example, if you want partition 1 of topic XXX to move to brokers 5 and 6, you could create a json file (manual-reassign.json) such as:

{"version":1,"partitions":[{"topic":"XXX","partition":1,"replicas":[5,6]}]}

启动方式与以前的方法相同:

The way it's launched is the same as in the previous method:

bin/kafka-reassign-partitions.sh --zookeeper $ZK_HOSTS 
 --reassignment-json-file manual-reassign.json --execute

这篇关于在某些代理关闭后，如何更改主题领导者或删除分区?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！