Kafka是否可以使用ANY配置参数值来保证单个分区内的消息排序? [英] Does Kafka guarantee message ordering within a single partition with ANY config param values?
问题描述
如果我在Producer上将Kafka配置参数设置为:
If I set Kafka config param at Producer as:
1. retries = 3
2. max.in.flight.requests.per.connection = 5
那么一个分区中的邮件可能不在send_order中.
then its likely that Messages within one partition may not be in send_order.
Kafka是否采取任何其他措施来确保分区中的消息仅保持发送顺序 或者 通过以上配置，是否可能在分区内出现乱序消息?
Does Kafka takes any extra step to make sure that messages within a partition remains in sent order only OR With above configuration, its possible to have out of order messages within a partition ?
推荐答案
不幸的是，没有.
在您当前的配置下，由于您的retries和max.in.flight.requests.per.connection设置，有机会出现消息无序的情况.
With your current configuration, there is a chance message will arrive unordered because of your retries and max.in.flight.requests.per.connection settings..
在retries配置设置为大于0的情况下，您将在以下情况下失去排序(只是一个带有随机数的示例):
With retries config set to greater than 0 you will lose ordering in the following case (just an example with random numbers):
- 您向位于代理0上的分区0发送消息/批处理，并且代理1和2是ISR.
- 经纪人0失败，经纪人1成为负责人.
- 您的消息/批次返回失败，需要重试.
- 同时，您将下一条消息/批处理发送到分区0，该分区现在已知位于代理1上，并且发生在实际重试上一个批处理之前.
- 消息/批次2得到确认(成功).
- 消息/批次1已发送，现在也得到确认.
- 订单丢失.
我可能是错的，但是在这种情况下，即使将max.in.flight.requests.per.connection设置为1，也可能会发生重新排序，如果出现代理故障转移，例如，可能会丢失消息顺序.可以比之前失败的批次更早地将批处理发送给代理，因为它也应该发送给该代理.
I might be wrong, but in this case, reordering can probably happen even with max.in.flight.requests.per.connection set to 1 you can lose message order in case of broker failover, e.g. batch can be sent to the broker earlier than the previous failed batch figures out it should go to that broker too.
关于将max.in.flight.requests.per.connection和retries设置在一起，这甚至更简单-如果您有多个未确认的对代理的请求，则第一个失败的请求将无序到达.
Regarding max.in.flight.requests.per.connection and retries being set together it's even simpler - if you have multiple unacknowledged requests to a broker, the first one to fail will arrive unordered.
但是，请考虑到这仅与消息/批处理由于某些原因而无法确认(发送给错误的经纪人，经纪人死亡等)的情况有关
However, please take into account this is only relevant to situations where a message/batch fails to acknowledge for some reason (sent to wrong broker, broker died etc.)
希望这会有所帮助
这篇关于Kafka是否可以使用ANY配置参数值来保证单个分区内的消息排序?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
