Kafka-消息订购保证 [英] Kafka - Message Ordering Guarantees

问题描述

关于订购，我遇到了两个短语

I come across two phrases with respect to ordering,

生产者发送到特定主题分区的
1. 消息将是按发送顺序附加.即，如果发送了记录M1由同一生产者作为记录M2，并且先发送M1，然后发送M1偏移量将小于M2，并在日志中更早显示.

1. Messages sent by a producer to a particular topic partition will be appended in the order they are sent. That is, if a record M1 is sent by the same producer as a record M2, and M1 is sent first, then M1 will have a lower offset than M2 and appear earlier in the log.

另一个

1. (config param)max.in.flight.requests.per.connection-最大数量客户端将在单个连接上发送的未确认请求阻止之前.请注意，如果此设置设置为大于1并且发送失败，存在消息重新排序的风险由于重试(即是否启用了重试).

1. (config param) max.in.flight.requests.per.connection - The maximum number of unacknowledged requests the client will send on a single connection before blocking. Note that if this setting is set to be greater than 1 and there are failed sends, there is a risk of message re-ordering due to retries (i.e., if retries are enabled).

问题是，如果发送失败，如#2所述，命令是否仍将保留在特定分区上?如果一条消息可能存在问题，则以下所有消息将被按顺序保留"以删除每个分区，或者将发送正确"消息并将失败的消息通知给应用程序?

The question is, will the order still be retained to a particular partition if there are failed sends like mentioned #2 ? if there is a potential issue with one message , all the following messages will be dropped "to retain the order" per partition or the "correct" messages will be sent and failed messages will be notified to the application ?

推荐答案

如果发送失败(如上述#2)，订单是否仍将保留到特定分区?"

"will the order still be retained to a particular partition if there are failed sends like mentioned #2?"

如您所复制的文档部分中所述，可能会更改顺序.

As written in the documentation part you have copied, there is a risk that the ordering is changed.

想象一下，您有一个主题，例如一个分区.您将 retries 设置为100，并将 max.in.flight.requests.per.connection 设置为5(大于1).请注意，只有将 acks 设置为1或"all"，重试才有意义.

Imagine, you have a topic with e.g. one partition. You set the retries to 100 and the max.in.flight.requests.per.connection to 5 which is greater than one. As a note, retries will only make sense if you set the acks to 1 or "all".

如果您打算按照K1，K2，K3，K4，K5的顺序生成以下消息，那么您的生产者就需要花费一些时间

If you plan to produce the following messages in the order K1, K2, K3, K4, K5 and it takes your producer some time to

• 实际创建批次并
• 向经纪人提出请求，并
• 等待经纪人的确认

您最多可以并行处理5个请求(基于 max.in.flight.request.per.connection 的设置).现在，生产出"K3".遇到了一些问题，并且进入了重试循环，可以在请求已经执行时生成消息K4和K5.

you could have up to 5 requests in parallel (based on the setting of max.in.flight.request.per.connection). Now, producing "K3" has some issues and it goes into the retry-loop, the messages K4 and K5 can be produced as the request was already in flight.

您的主题最终将以以下顺序显示消息:K1，K2，K4，K5，K3.

Your topic would end up with messages in that order: K1, K2, K4, K5, K3.

如果您在Kafka Producer中启用幂等，则仍然可以按照

In case you enable idempotency in the Kafka Producer, the ordering would still be guaranteed as explained in Ordering guarantees when using idempotent Kafka Producer

这篇关于Kafka-消息订购保证的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！