Kafka 配置 min.insync.replicas 不起作用 [英] Kafka configuration min.insync.replicas not working

问题描述

这是我学习 kafka 的早期阶段.我正在检查本地机器中的每个 kafka 属性/概念.

所以我遇到了这个属性min.insync.replicas，这是我的理解.如果我误解了什么，请纠正我.

• 将消息发送到主题后，必须将该消息写入至少 min.insync.replicas 数量的关注者.
• min.insync.replicas 还包括领导者.
• 如果可用的 live brokers(indirectly, in sync replicas )少于指定的 min.insync.replicas ，那么生产者将引发无法发布的异常消息.

以下是我创建上述场景所遵循的步骤

1. 在本地启动了 3 个代理，代理 ID 为 0、1 和 2
2. 创建主题 insync 并将 min.insync.replicas 设置为 2使用以下命令

sudo ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic insync --config min.insync.replicas=2

3. 描述导致以下结果的主题

<块引用>

主题:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2主题:insync 分区:0 Leader:2 Replicas:2,0,1 Isr:1,2,0

此时，我确保我提供的属性是由 kafka 选择的

4. 我开始使用以下命令从终端发送消息并使用它们

   生产者:./kafka-console-producer.sh --broker-list localhost:9092 --topic insync --producer.config ../config/producer.properties

   消费者:./kafka-console-consumer.sh --zookeeper localhost:2181 --topic insync

此时，我能够成功发送和接收消息.

5. 购买了 2 个经纪人(0 和 2)并描述了主题并导致了以下内容

<块引用>

主题:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2主题:insync 分区:0 Leader:1 Replicas:2,0,1 Isr:1

此时，In Sync Replicas 只有 1(Isr: 1)

然后我尝试生成消息并且它起作用了.我能够从控制台生产者发送消息，我可以在控制台消费者中看到这些消息.

我的 Kafka 版本:kafka_2.10-0.10.0.0

以下是生产者属性:

<块引用>

bootstrap.servers=localhost:9092

compression.type=none

batch.size=20

acks=all

我预计生产者会以 NotEnoughReplicasException 失败，如 this.

<块引用>

公共类NotEnoughReplicasException扩展 RetriableException

分区的同步副本数低于>min.insync.replicas

但它工作正常.

我错过了什么吗?如何创建场景?

*************** 编辑 **********************

我没有从控制台生成器生成消息，而是尝试从 Java 代码生成消息.这一次，我在 kafka broker 中得到了预期的异常.虽然我在生产者(java代码)中期望它.由于这个实验提出了更多问题，我发布了另一个问题.

解决方案

acks 是否设置为全部"?如果没有，请尝试将其设置为全部

Its my early days in learning kafka. And I am checking out every kafka property/concept in my local machine.

So I came across this property min.insync.replicas and here is my understanding. Please correct me if I've misunderstood anything.

• Once a message is sent to a topic, the message must be written to at least min.insync.replicas number of followers.
• min.insync.replicas also includes the leader.
• If number of available live brokers( indirectly, in sync replicas ) are less than the specified min.insync.replicas , then producer will raise an exception failing to publish the message.

Following are the steps I followed to create the above scenario

1. Started 3 brokers in local with broker Ids 0, 1 and 2
2. created the topic insync and set min.insync.replicas to 2 using the following command

sudo ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic insync --config min.insync.replicas=2

3. Describe the topic resulted in the following

Topic:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2 Topic: insync Partition: 0 Leader: 2 Replicas: 2,0,1 Isr: 1,2,0

At this point, I made sure the property I've provided is picked by kafka

4. I started sending messages and consuming them from terminal using following command

   Producer: ./kafka-console-producer.sh --broker-list localhost:9092 --topic insync --producer.config ../config/producer.properties

   Consumer: ./kafka-console-consumer.sh --zookeeper localhost:2181 --topic insync

At this point, I was able to send and receive messages successfully.

5. Bought down 2 brokers (0 and 2) and described the topic and resulted in following

Topic:insync PartitionCount:1 ReplicationFactor:3 Configs:min.insync.replicas=2 Topic: insync Partition: 0 Leader: 1 Replicas: 2,0,1 Isr: 1

At this point, the In Sync Replicas are just 1(Isr: 1)

Then I tried to produce the message and it worked. I was able to send messages from console-producer and I could see those messages in console consumer.

My Kafka version: kafka_2.10-0.10.0.0

following are the producer properties:

bootstrap.servers=localhost:9092

compression.type=none

batch.size=20

acks=all

I expected the producer to fail with NotEnoughReplicasException as mentioned in this.

public class NotEnoughReplicasException extends RetriableException

Number of insync replicas for the partition is lower than >min.insync.replicas

but it worked normally.

Am I missing something? How can I create the scenario?

*************** EDIT **********************

Instead of producing the messages from console producer, I tried to generate messages from java code. This time, I got the expected exception in the kafka broker. Although I expected it in the producer (java code). As this experiment is raising more questions, I've posted another question.

解决方案

is acks set to "all"? if not, try setting it to all

这篇关于Kafka 配置 min.insync.replicas 不起作用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！