Kafka:如何从副本集中删除代理 [英] Kafka: how to remove broker from Replica Set

问题描述

我有一个包含以下副本的分区:

I have a partition with the following Replicas:

Topic: topicname    Partition: 10   Leader: 1   Replicas: 1,2,4,3   Isr: 1,2,3

其中副本 4 是不存在的代理.我不小心将这个代理添加到副本集中作为一个错字.

Where Replica 4 is a non-existent broker. I accidentally added this broker into the replica set as a typo.

我想从副本集中删除 4 个.但是在运行 kafka-reassign-partitions.sh 之后，删除副本 #4 的重新分配从未完成.

I want to remove 4 from the Replica set. but after running kafka-reassign-partitions.sh, the reassignment to remove Replica #4 never finishes.

kafka-reassign-partitions.sh --zookeeper myzookeeperhost:2181 --reassignment-json-file remove4.txt --execute 

remove4.txt 的样子

Where remove4.txt looks like

{ "partitions": [
     { "topic": "topicname", "partition": 2, "replicas": [1,2,3] }
], "version": 1 }

重新分配卡住了:

kafka-reassign-partitions.sh --zookeeper myzookeeperhost:2181 --reassignment-json-file remove4.txt --verify
Status of partition reassignment:
Reassignment of partition [topicname,10] is still in progress

我检查了控制器日志，看起来重新分配命令被拾取，但之后没有任何反应:

I checked the controller log, it looks like the reassignment command was picked up, but nothing happens afterwards:

[2017-08-01 06:46:07,653] DEBUG [PartitionsReassignedListener on 101 (the controller broker)]: Partitions reassigned listener fired for path /admin/reassign_partitions. Record partitions to be reassigned {"version":1,"partitions":[{"topic":"topicname","partition":10,"replicas":[1,2,3]}]} (kafka.controller.PartitionsReassignedListener)

对我做错了什么有什么想法吗?如何从副本集中删除代理 #4?更新:我正在运行 kafka 10

Any ideas on what I'm doing wrong? How to I remove broker #4 from the replica set? update: I'm running kafka 10

推荐答案

我能够通过启动一个新的代理来解决这个问题，该代理的代理 ID 与添加的代理 ID 匹配(在您的情况下为 4).

I was able to solve this issue by spinning up a new broker with a broker-id matching the one that was added (in your case 4).

>

Kafka 快速入门 指南向您展示了如何使用特定 ID 启动代理.使用 id 4 启动节点后，运行:

The Kafka Quickstart guide shows you how to spin up a broker with a specific id. Once you have your node up with id 4, run:

./bin/kafka-topics.sh --zookeeper localhost:2181 --topic badbrokertest --describe

您应该看到所有副本都在 isr 列中，如下所示:

You should see that all the replicas are in the isr column like so:

Topic:badbrokertest PartitionCount:3    ReplicationFactor:3 Configs:
Topic: badbrokertest    Partition: 0    Leader: 1   Replicas: 1,2,3 Isr: 1,2,3
Topic: badbrokertest    Partition: 1    Leader: 1   Replicas: 1,2,3 Isr: 1,2,3  
Topic: badbrokertest    Partition: 2    Leader: 1   Replicas: 1,2,3,4   Isr: 1,2,3,4

现在您可以重新分配分区了！

Now you can reassign your partitions!

./bin/kafka-reassign-partitions.sh --reassignment-json-file badbroker2.json  --zookeeper localhost:2181

badbroker2.json 如下所示:

Where badbroker2.json looks like:

{
    "version":1,
    "partitions":[
        {"topic":"badbrokertest","partition":0,"replicas":[1,2,3]},
        {"topic":"badbrokertest","partition":1,"replicas":[1,2,3]},
        {"topic":"badbrokertest","partition":2,"replicas":[1,2,3]}
    ]
}

简而言之，一旦您通过添加缺少的代理同步了所有副本，您就可以删除不需要的代理.

So in short, once you've sync'd all your replicas by adding the missing broker you can remove the unneeded broker.

如果您使用多个服务器，请务必在配置中设置 listeners 字段，以使您的临时代理可用于其他代理.快速入门指南不考虑这种情况.

If you're working with several servers, be sure to set the listeners field in the config to make your temporary broker available to the others brokers. The Quickstart guide doesn't consider that case.

listeners=PLAINTEXT://10.9.1.42:9093

这篇关于Kafka:如何从副本集中删除代理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！