卡夫卡新生产者超时 [英] Kafka new producer timeout

问题描述

我正在使用新的 kafka 生产者客户端并将 timeout.ms 属性设置为 50 毫秒.

I'm using the new kafka producer client and set the timeout.ms property to 50 ms.

这里是生产者使用的完整配置:

Here is the complete configuration used in the producer:

props.put("acks", "1");
props.put("buffer.memory", "33554432");
props.put("retries", "1");
props.put("batch.size", "16384");
props.put("client.id", "foo");
props.put("linger.ms", "0");
props.put("timeout.ms", "50");

某些高负载时刻的请求平均响应时间为 4 秒，但我没有收到任何超时错误.

The request average response time in some moments of high load is 4 seconds, but I don't get any timeout error.

有人知道这个超时是怎么计算的，什么时候开始计数，什么时候结束?有没有办法配置一个超时时间，从生产者的send方法被调用的那一刻开始?

Does Someone know how this timeout is calculated, when it begins to be counted and when it finishes? Is there a way to configure a timeout to start from the moment the producer's send method is called?

推荐答案

新的 timeout.ms 属性适用于生产者的 ack 配置.例如考虑以下情况

The new timeout.ms property works with the ack configuration of the producer. For example consider the following situation

ack = all
timeout.ms = 3000

在这种情况下 ack = all 意味着领导者在收到完整的同步副本 (ISR) 集的确认之前不会响应，并且获得此确认的最大等待时间为3000 毫秒.如果在给定时间内没有收到预期数量的确认，它将返回一个错误.

In this case ack = all means that the leader will not respond untill it receives acknowledgement for the full set of in-sync replicas (ISR) and the maximum wait time to get this acknowledgement will be 3000 ms. If it didn't receive the expected number of acknowledgements within the given time it will return an error.

另请注意，此属性不考虑网络延迟.

Also note this property does not consider network latency.

来自文档页面:

该配置控制服务器等待来自追随者的确认以满足生产者使用 acks 配置指定的确认要求的最长时间.如果超时后未满足请求的确认数量，则将返回错误.此超时是在服务器端测量的，不包括请求的网络延迟.

The configuration controls the maximum amount of time the server will wait for acknowledgments from followers to meet the acknowledgment requirements the producer has specified with the acks configuration. If the requested number of acknowledgments are not met when the timeout elapses an error will be returned. This timeout is measured on the server side and does not include the network latency of the request.

因此，在您使用 ack=1 的情况下(我不是 100% 确定这一点，如果适用，可以进行任何更正)，如果领导者无法响应(在将记录写入其自己的日志而不等待所有关注者的完全确认)应在 50 毫秒内抛出错误.

So in your case with ack=1 (I am not 100% sure about this and open to any correction if applicable) if the leader is not able to respond (after writing the record to its own log without awaiting full acknowledgement from all followers) within 50ms should throw an error.

这篇关于卡夫卡新生产者超时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！