Kafka:如何从副本集中删除代理 [英] Kafka: how to remove broker from Replica Set

问题描述

我有一个带有以下副本的分区:

I have a partition with the following Replicas:

Topic: topicname    Partition: 10   Leader: 1   Replicas: 1,2,4,3   Isr: 1,2,3

其中副本服务器4不存在的代理.我不小心将此错字添加到副本集作为错字.

Where Replica 4 is a non-existent broker. I accidentally added this broker into the replica set as a typo.

我要从副本集中删除4个.但是在运行 kafka-reassign-partitions.sh 之后，删除副本4的重新分配从未完成.

I want to remove 4 from the Replica set. but after running kafka-reassign-partitions.sh, the reassignment to remove Replica #4 never finishes.

kafka-reassign-partitions.sh --zookeeper myzookeeperhost:2181 --reassignment-json-file remove4.txt --execute 

remove4.txt的外观

Where remove4.txt looks like

{ "partitions": [
     { "topic": "topicname", "partition": 2, "replicas": [1,2,3] }
], "version": 1 }

重新分配卡住了:

kafka-reassign-partitions.sh --zookeeper myzookeeperhost:2181 --reassignment-json-file remove4.txt --verify
Status of partition reassignment:
Reassignment of partition [topicname,10] is still in progress

我检查了控制器日志，看起来好像收到了重新分配命令，但是此后什么也没发生:

I checked the controller log, it looks like the reassignment command was picked up, but nothing happens afterwards:

[2017-08-01 06:46:07,653] DEBUG [PartitionsReassignedListener on 101 (the controller broker)]: Partitions reassigned listener fired for path /admin/reassign_partitions. Record partitions to be reassigned {"version":1,"partitions":[{"topic":"topicname","partition":10,"replicas":[1,2,3]}]} (kafka.controller.PartitionsReassignedListener)

关于我在做什么错的任何想法?如何从副本集中删除代理#4?更新:我正在运行kafka 10

Any ideas on what I'm doing wrong? How to I remove broker #4 from the replica set? update: I'm running kafka 10

推荐答案

我能够通过解决一个新经纪人的问题，该经纪人的经纪人ID与添加的经纪人ID相匹配(在您的案例4中).

I was able to solve this issue by spinning up a new broker with a broker-id matching the one that was added (in your case 4).

>

Kafka快速入门指南向您展示了如何启动具有特定ID的经纪人.将节点的ID设置为4后，运行:

The Kafka Quickstart guide shows you how to spin up a broker with a specific id. Once you have your node up with id 4, run:

./bin/kafka-topics.sh --zookeeper localhost:2181 --topic badbrokertest --describe

您应该看到所有副本都在 isr 列中，如下所示:

You should see that all the replicas are in the isr column like so:

Topic:badbrokertest PartitionCount:3    ReplicationFactor:3 Configs:
Topic: badbrokertest    Partition: 0    Leader: 1   Replicas: 1,2,3 Isr: 1,2,3
Topic: badbrokertest    Partition: 1    Leader: 1   Replicas: 1,2,3 Isr: 1,2,3  
Topic: badbrokertest    Partition: 2    Leader: 1   Replicas: 1,2,3,4   Isr: 1,2,3,4

现在您可以重新分配分区了！

Now you can reassign your partitions!

./bin/kafka-reassign-partitions.sh --reassignment-json-file badbroker2.json  --zookeeper localhost:2181

badbroker2.json如下所示:

Where badbroker2.json looks like:

{
    "version":1,
    "partitions":[
        {"topic":"badbrokertest","partition":0,"replicas":[1,2,3]},
        {"topic":"badbrokertest","partition":1,"replicas":[1,2,3]},
        {"topic":"badbrokertest","partition":2,"replicas":[1,2,3]}
    ]
}

简而言之，一旦您通过添加缺少的代理来同步所有副本，就可以删除不需要的代理.

So in short, once you've sync'd all your replicas by adding the missing broker you can remove the unneeded broker.

如果要使用多台服务器，请确保在配置中设置侦听器字段，以使其他代理可以使用您的临时代理.快速入门指南不考虑这种情况.

If you're working with several servers, be sure to set the listeners field in the config to make your temporary broker available to the others brokers. The Quickstart guide doesn't consider that case.

listeners=PLAINTEXT://10.9.1.42:9093

这篇关于Kafka:如何从副本集中删除代理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！