Kafka 是否保证具有任何配置参数值的单个分区内的消息排序? [英] Does Kafka guarantee message ordering within a single partition with ANY config param values?
问题描述
如果我将 Producer 的 Kafka 配置参数设置为:
If I set Kafka config param at Producer as:
1. retries = 3
2. max.in.flight.requests.per.connection = 5
那么很可能一个分区内的消息可能不在 send_order 中.
then its likely that Messages within one partition may not be in send_order.
Kafka 是否采取任何额外步骤来确保分区内的消息仅保持发送顺序或者通过以上配置，是否有可能在一个分区内有乱序的消息?
Does Kafka takes any extra step to make sure that messages within a partition remains in sent order only OR With above configuration, its possible to have out of order messages within a partition ?
推荐答案
很遗憾，没有.
根据您当前的配置，由于您的重试和max.in.flight.requests.per.connection设置，消息可能会无序到达..
With your current configuration, there is a chance message will arrive unordered because of your retries and max.in.flight.requests.per.connection settings..
如果 retries 配置设置为大于 0，您将在以下情况下失去排序(只是一个随机数示例):
With retries config set to greater than 0 you will lose ordering in the following case (just an example with random numbers):
- 您向位于代理 0 上的分区 0 发送消息/批处理，代理 1 和 2 是 ISR.
- 代理 0 失败，代理 1 成为领导者.
- 您的消息/批次返回失败，需要重试.
- 与此同时，您将下一条消息/批次发送到现在已知位于代理 1 上的分区 0，这发生在您的上一批实际重试之前.
- 消息/批次 2 得到确认(成功).
- 消息/批次 1 被重新发送，现在也得到确认.
- 订单丢失.
我可能错了，但在这种情况下，即使 max.in.flight.requests.per.connection 设置为 1，也可能会发生重新排序，您可能会在代理故障转移的情况下丢失消息顺序，例如批次可以在前一个失败的批次确定它也应该发送给该代理之前发送给代理.
I might be wrong, but in this case, reordering can probably happen even with max.in.flight.requests.per.connection set to 1 you can lose message order in case of broker failover, e.g. batch can be sent to the broker earlier than the previous failed batch figures out it should go to that broker too.
关于 max.in.flight.requests.per.connection 和 retries 设置在一起，它甚至更简单 - 如果您有多个未确认的请求到代理，第一个一个失败将无序到达.
Regarding max.in.flight.requests.per.connection and retries being set together it's even simpler - if you have multiple unacknowledged requests to a broker, the first one to fail will arrive unordered.
但是，请注意这仅与消息/批次由于某种原因(发送给错误的经纪人、经纪人死亡等)而未能确认的情况相关
However, please take into account this is only relevant to situations where a message/batch fails to acknowledge for some reason (sent to wrong broker, broker died etc.)
希望能帮到你
这篇关于Kafka 是否保证具有任何配置参数值的单个分区内的消息排序?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
