Kafka Producer配置重试策略 [英] Kafka Producer config retry strategy

问题描述

需要更改Kafka Producer配置的哪些参数，以便生产者应:1)重试n次2)在n个间隔之后如果代理关闭，则显示相同的消息.

What parameters of Kafka Producer config are needed to be changed so that the producer should: 1) Retry n times 2) After n interval for the same message in case if the broker is down.

我需要处理与此有关的情况: https://github.com/rsyslog/rsyslog/issues/1052

I need to handle a situation related to this: https://github.com/rsyslog/rsyslog/issues/1052

推荐答案

您可以将"重试"设置为n(次数).但这还不够，您还需要研究其他配置，否则可能会因此而受到影响或使配置无效.

You can set "retries" to n(number of times). But it is not enough you need to look into other configurations also which might get affected because of this or not making it effective.

1)如果生产者的 acks = 0 ，则它将无法正常工作.因为acks = 0→生产者不等待任何形式的确认.在这种情况下，不能保证经纪人已收到该记录.重试配置未生效，因为无法知道是否发生任何故障.

1) if you have acks = 0 for producer then it will not work. Because acks = 0 → The producer does not wait for any kind of acknowledgment. In this case, no guarantee can be made that the record was received by the broker. retries config does not take effect as there is no way to know if any failure occurred.

2)如果您正在寻找事件的有序交付，则需要将 max.in.flight.requests.per.connection 设置为1.

2) if you are looking for ordered delivery of event(s) then you need to set max.in.flight.requests.per.connection to 1.

3) retry.backoff.ms -尝试重试对给定主题分区的失败请求之前要等待的时间.这样可以避免在某些失败情况下，在紧密的循环中重复发送请求.

3) retry.backoff.ms - The amount of time to wait before attempting to retry a failed request to a given topic partition. This avoids repeatedly sending requests in a tight loop under some failure scenarios.

4) request.timeout.ms -配置控制客户端等待请求响应的最长时间.如果超时之前仍未收到响应，则客户端将在必要时重新发送请求，如果重试用尽，则客户端将使请求失败.

4) request.timeout.ms - The configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted.

5) delivery.timeout.ms -调用send()返回后报告成功或失败时间的上限.这限制了记录在发送之前将被延迟的总时间，等待来自代理的确认的时间(如果期望)以及允许可重发的发送失败的时间.如果遇到不可恢复的错误，重试已用尽或记录将添加到已达到较早的交货到期期限的批次中.

5) delivery.timeout.ms - An upper bound on the time to report success or failure after a call to send() returns. This limits the total time that a record will be delayed prior to sending, the time to await acknowledgment from the broker (if expected), and the time allowed for retriable send failures. The producer may report failure to send a record earlier than this config if either an unrecoverable error is encountered, the retries have been exhausted, or the record is added to a batch which reached an earlier delivery expiration deadline.

参考链接: https://kafka.apache.org/documentation/#producerconfigs

这篇关于Kafka Producer配置重试策略的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！