创建Kafka主题不会导致任何领导者 [英] Creating a Kafka topic results in no leader

问题描述

我正在使用Kafka v0.9.0.1(Scala v2.11)和com.101tec:zkclient v0.7.我正在尝试使用AdminUtils创建一个kafka主题.我的代码如下.

I am using Kafka v0.9.0.1 (Scala v2.11) and the com.101tec:zkclient v0.7. I am trying to use AdminUtils to create a kafka topic. My code is the following.

String zkServers = "node1:2181,node2:2181,node3:2181,node4:2181";
Integer sessionTimeout = (int)TimeUnit.SECONDS.toMillis(10L);
Integer connectionTimeout = (int)TimeUnit.SECONDS.toMillis(8L);
ZkSerializer zkSerializer = ZKStringSerializer$.MODULE$;
Boolean isSecureKafkaCluster = false;
String topic = "test";
Integer partitions = 1;
Integer replication = 3;

ZkClient zkClient = new ZkClient(zkServers, sessionTimeout, connectionTimeout, zkSerializer);
ZkUtils zkUtils = new ZkUtils(zkClient, new ZkConnection(zkServers), isSecureKafkaCluster)
if(!AdminUtils.topicExists(zkUtils, topic)) {
 AdminUtils.createTopic(zkUtils, topic, partitions, replications, new Properties());
}

该主题实际上是通过以下命令验证后创建的.

The topic is actually created as verified by the following command.

bin/kafka-topics.sh --describe --zookeeper node1:2181 --topic test

但是，输出不符合预期.

However, the output is not as expected.

Topic:test  PartitionCount:1    ReplicationFactor:1 Configs:
Topic: test Partition: 0    Leader: -1  Replicas: 4 Isr: 

如果我使用脚本.

bin/kafka-topics.sh --create --zookeeper node1:2181 --replication-factor 3 --partitions 1 --topic topic1

然后我看到以下内容.

Topic:test1 PartitionCount:1    ReplicationFactor:3 Configs:
Topic: test1    Partition: 0    Leader: 2   Replicas: 2,3,4 Isr: 2

关于我在做什么错的任何想法?效果是，如果我使用Producer向主题发送ProducerRecord，则该主题上没有任何显示.

Any ideas on what I'm doing wrong? The effect is that if I use a Producer to send a ProducerRecord to the topic, nothing shows up on the topic.

推荐答案

我遇到了同样的问题.
解决方案:

I had the same issue.
Solution:

1. 清除zk元信息(/经纪人/主题)

1. Clean zk meta info (/brokers/topic)

清除所有/data目录以删除属于该主题的所有主题分区文件夹

Clean all /data dir to remove all topic-partition folders belongs to that topic

立即重新启动所有代理的整个kafka集群.

Restart the whole kafka cluster all brokers at once.

重新创建该主题.

这解决了我的问题.而且我认为根本原因是kafka本身无法处理干净删除主题的缺陷(自v1.0.0起已修复).

This solved my problem. And I think the root cause was the defect from kafka itself failing to handle clean removal topics (this has been fixed since v1.0.0).

即使使用Kafka(> = v1.0.0)，如果要删除一个空主题或您的kafka集群负载过大，有时删除主题也会卡住.
解决方案就像重新启动控制器代理一样简单. (您始终可以通过get /controller在ZK:/controller下找到控制器代理).因此，只需重新启动一个代理即可，而不是重新启动整个kafka群集.

even with Kafka(>= v1.0.0), sometimes deleting topic will stuck if you are deleting an empty topic or if your kafka cluster is under extreme load.
solution would be as simple as restarting the controller broker. (you can always find the controller broker under ZK: /controller by get /controller). so just restarting one broker instead of the whole kafka cluster.

这篇关于创建Kafka主题不会导致任何领导者的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！