kafka ack = all and min-isr [英] kafka ack=all and min-isr

问题描述

摘要

Kafka的文档和代码注释建议，当生产者设置 acks 设置为 all 时，只有在 all时，才会将确认发送给生产者同步副本已经赶上了，但是代码( Partition.Scala ， checkEnoughReplicasReachOffset )似乎建议尽快将确认发送给最小的同步副本已经赶上了.

The docs and code comments for Kafka suggest that when the producer setting acks is set to all then an ack will only be sent to the producer when all in-sync replicas have caught up, but the code (Partition.Scala, checkEnoughReplicasReachOffset) seems to suggest that the ack is sent as soon as min in-sync replicas have caught up.

详细信息

kafka文档具有以下内容:

The kafka docs have this:

acks = all这意味着领导者将等待完整的同步副本集确认记录.来源

还要查看Kafka源代码- partition.scala checkEnoughReplicasReachOffset()有以下注释(强调我的意思):

Also, looking at the Kafka source code - partition.scala checkEnoughReplicasReachOffset() has the following comment (emphasis mine):

请注意，只有在requiredAcks = -1且我们正在等待ISR中的所有副本被完全捕获到与该生产请求相对应的(本地)领导者的偏移量之前，才会调用此方法我们确认了农产品的要求.

Note that this method will only be called if requiredAcks = -1 and we are waiting for all replicas in ISR to be fully caught up to the (local) leader's offset corresponding to this produce request before we acknowledge the produce request.

最后，关于堆栈溢出的答案(再次强调我的想法)

Finally, this answer on Stack Overflow (emphasis mine again)

此外，最小同步副本设置还指定为使分区保持可写状态而需要同步的最小副本数.当生产者指定ack(-1/all config)时，它仍将在那一刻等待所有同步副本中的确认(与最小同步副本的设置无关).

Also the min in-sync replica setting specifies the minimum number of replicas that need to be in-sync for the partition to remain available for writes. When a producer specifies ack (-1 / all config) it will still wait for acks from all in sync replicas at that moment (independent of the setting for min in-sync replicas).

但是当我查看Partition.Scala中的代码时(请注意 minIsr< curInSyncReplicas.size ):

But when I look at the code in Partition.Scala (note minIsr < curInSyncReplicas.size):

def checkEnoughReplicasReachOffset(requiredOffset: Long): (Boolean, Errors) = {
  ...
  val minIsr = leaderReplica.log.get.config.minInSyncReplicas
  if (leaderReplica.highWatermark.messageOffset >= requiredOffset) {          
    if (minIsr <= curInSyncReplicas.size)
      (true, Errors.NONE)

调用此代码的代码将返回确认:

The code that calls this returns the ack:

if (error != Errors.NONE || hasEnough) {
  status.acksPending = false
  status.responseStatus.error = error
}

因此，一旦异步副本集大于最小同步副本，该代码看起来就会返回一个ack.但是，文档和注释建议仅在所有同步副本都已捕获后才发送ack.我想念什么?至少， checkEnoughReplicasReachOffset 上方的注释看起来应该更改.

So, the code looks like it returns an ack as soon as the in-sync replica set are greater than min in-sync replicas. However, the documentation and comments suggest that the ack is only sent once all in-sync replicas have caught up. What am I missing? At the very least, the comment above checkEnoughReplicasReachOffset looks like it should be changed.

推荐答案

感谢jira-dev邮件列表中的Ismael.

Thanks to Ismael on the jira-dev mailing list.

关键是直线:

if(leaderReplica.highWatermark.messageOffset> = requiredOffset){

仅当ISR中的所有副本具有特定偏移时，高水位标记才会移动.

The high watermark only moves when all the replicas in ISR have that particular offset.

这篇关于kafka ack = all and min-isr的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！