Kafka配置min.insync.replicas不起作用 [英] Kafka configuration min.insync.replicas not working

问题描述

这是我早期学习卡夫卡的日子.而且我正在检查本地计算机上的每个kafka属性/概念.

因此，我遇到了此属性min.insync.replicas，这是我的理解.如果我误解了任何内容，请纠正我.

• 一旦将消息发送到主题，该消息必须至少写入min.insync.replicas个关注者.
• min.insync.replicas还包括领导者.
• 如果可用的实时经纪人(间接同步复制中的 )数量少于指定的min.insync.replicas，则生产者将引发一个异常，无法发布消息.

以下是我创建上述场景所遵循的步骤

1. 在本地启动了3个经纪人，其经纪人ID为0、1和2
2. 创建主题 异步 ，并将min.insync.replicas设置为2 使用以下命令

sudo ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic insync --config min.insync.replicas=2

3. 描述导致以下问题的主题

主题:insync PartitionCount:1复制因子:3配置:min.insync.replicas = 2 主题:insync分区:0领导者:2复制副本:2,0,1 Isr:1,2,0

在这一点上，我确保我提供的属性被kafka选中

4. 我开始使用以下命令从终端发送消息并使用它们

   制作人:./kafka-console-producer.sh --broker-list localhost:9092 --topic insync --producer.config ../config/producer.properties

   消费者:./kafka-console-consumer.sh --zookeeper localhost:2181 --topic insync

在这一点上，我能够成功发送和接收消息.

5. 买断了2个经纪人(0个和2个)并描述了该主题，并导致了以下

主题:insync PartitionCount:1复制因子:3配置:min.insync.replicas = 2 主题:insync分区:0领导者:1复制副本:2,0,1 Isr:1

这时，同步副本只有1( Isr:1 )

然后，我尝试生成该消息，然后它起作用了.我能够从控制台生产者发送消息，并且可以在控制台使用者中看到这些消息.

我的Kafka版本: kafka_2.10-0.10.0.0

以下是生产者属性:

bootstrap.servers = localhost:9092

compression.type = none

batch.size = 20

acks = all

如NotEnoughReplicasException而失败. ="nofollow noreferrer">此.

公共类NotEnoughReplicasException 扩展RetriableException

该分区的异步副本数少于> min.insync.replicas

但是它正常工作.

我错过了什么吗?如何创建方案?

**************** 编辑 **********************

我没有从控制台生产者那里产生消息，而是尝试从Java代码产生消息.这次，我在kafka经纪人中得到了预期的例外.尽管我期望在生产者中使用它(Java代码).随着该实验提出了更多问题，我发布了另一个问题.

解决方案

acks是否设置为"all"?如果没有，请尝试将其全部设置

Its my early days in learning kafka. And I am checking out every kafka property/concept in my local machine.

So I came across this property min.insync.replicas and here is my understanding. Please correct me if I've misunderstood anything.

• Once a message is sent to a topic, the message must be written to at least min.insync.replicas number of followers.
• min.insync.replicas also includes the leader.
• If number of available live brokers( indirectly, in sync replicas ) are less than the specified min.insync.replicas , then producer will raise an exception failing to publish the message.

Following are the steps I followed to create the above scenario

1. Started 3 brokers in local with broker Ids 0, 1 and 2
2. created the topic insync and set min.insync.replicas to 2 using the following command

sudo ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic insync --config min.insync.replicas=2

3. Describe the topic resulted in the following

Topic:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2 Topic: insync Partition: 0 Leader: 2 Replicas: 2,0,1 Isr: 1,2,0

At this point, I made sure the property I've provided is picked by kafka

4. I started sending messages and consuming them from terminal using following command

   Producer: ./kafka-console-producer.sh --broker-list localhost:9092 --topic insync --producer.config ../config/producer.properties

   Consumer: ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic insync

At this point, I was able to send and receive messages successfully.

5. Bought down 2 brokers (0 and 2) and described the topic and resulted in following

Topic:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2 Topic: insync Partition: 0 Leader: 1 Replicas: 2,0,1 Isr: 1

At this point, the In Sync Replicas are just 1(Isr: 1)

Then I tried to produce the message and it worked. I was able to send messages from console-producer and I could see those messages in console consumer.

My Kafka version: kafka_2.10-0.10.0.0

following are the producer properties:

bootstrap.servers=localhost:9092

compression.type=none

batch.size=20

acks=all

I expected the producer to fail with NotEnoughReplicasException as mentioned in this.

public class NotEnoughReplicasException extends RetriableException

Number of insync replicas for the partition is lower than >min.insync.replicas

but it worked normally.

Am I missing something? How can I create the scenario?

*************** EDIT **********************

Instead of producing the messages from console producer, I tried to generate messages from java code. This time, I got the expected exception in the kafka broker. Although I expected it in the producer (java code). As this experiment is raising more questions, I've posted another question.

解决方案

is acks set to "all"? if not, try setting it to all

这篇关于Kafka配置min.insync.replicas不起作用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！