Kafka Streams错误 - 分区上的偏移提交失败，请求超时 [英] Kafka Streams error - Offset commit failed on partition, request timed out

问题描述

我们使用Kafka Streams来消费，处理和制作消息，而在PROD环境中，我们遇到了多个主题的错误：

We use Kafka Streams for consuming, processing and producing messages, and on PROD env we faced with errors on multiple topics:

ERROR org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - [Consumer clientId=app-xxx-StreamThread-3-consumer, groupId=app] 
Offset commit failed on partition xxx-1 at offset 13920: 
The request timed out.[]

对于负载较小的主题，这些错误很少发生，但对于高负荷（和尖峰）的主题，每个主题每天会发生数十次错误。主题有多个分区（例如10个）。似乎这个问题不影响数据的处理（尽管性能），因为抛出异常（甚至可能是同一偏移的多个错误），消费者以后重新读取消息并成功处理它。

These errors occur rarely for topics with small load, but for topics with high load (and spikes) errors occur dozens of times a day per topic. Topics have multiple partitions (e.g. 10). Seems this issue does not affect processing of data (despite performance), as after throwing exception (even could be multiple errors for the same offset), consumer later re-read message and successfully process it.

我看到此错误消息出现在 kafka-clients 版本 1.0.0 中，原因是< a href =https://github.com/apache/kafka/commit/6896f1ddb7650f42630aef8c67c8b61866e9fc00#diff-e9c1ee46a19a8684d9d8d8a8c77f9005 =nofollow noreferrer> PR ，但在之前的 kafka-clients 相同用例的版本（消费者的 Errors.REQUEST_TIMED_OUT ）类似的消息（组{}的偏移提交失败：{使用 debug 级别记录了）。
对我来说，更新日志级别以警告此类用例更合乎逻辑。

I see that this error message appeared in kafka-clients version 1.0.0 due to PR, but in previous kafka-clients versions for the same use case (Errors.REQUEST_TIMED_OUT on consumer) similar message (Offset commit for group {} failed: {}) was logged with debug level. as for me, it would be more logical to update log level to warning for such use case.

如何解决此问题？可能是根本原因？也许更改消费者属性或分区设置可以帮助摆脱这样的问题。

How to fix this issue? What could be the root cause? Maybe changing consumer properties or partition setup could help to get rid of such issue.

我们使用以下实现来创建Kafka Streams：

we use the following implementation for creating Kafka Streams:

StreamsBuilder builder = new StreamsBuilder();
KStream<String, String> stream = builder.<String, String>stream(topicName);
stream.foreach((key, value) -> processMessage(key, value));
Topology topology = builder.build();
StreamsConfig streamsConfig = new StreamsConfig(consumerSettings);
new KafkaStreams(streamsTopology, streamsConfig);

我们的Kafka消费者设置：

our Kafka consumer settings:

bootstrap.servers: xxx1:9092,xxx2:9092,...,xxx5:9092
application.id: app
state.dir: /tmp/kafka-streams/xxx
commit.interval.ms: 5000       # also I tried default value 30000
key.serde: org.apache.kafka.common.serialization.Serdes$StringSerde
value.serde: org.apache.kafka.common.serialization.Serdes$StringSerde
timestamp.extractor: org.apache.kafka.streams.processor.WallclockTimestampExtractor

kafka经纪人版本： kafka_2.11-0.11.0.2 。两个版本的Kafka Streams都发生
错误： 1.0.1 和 1.1.0 。

kafka broker version: kafka_2.11-0.11.0.2. error occur on both versions of Kafka Streams: 1.0.1 and 1.1.0.

推荐答案

看起来您遇到Kafka集群问题，而Kafka消费者在尝试提交抵消时会超时。
您可以尝试为Kafka消费者增加与连接相关的配置

Looks like you have issue with Kafka cluster and Kafka consumer is get timed out while trying to commit offsets. You can try to increase connection related configs for Kafka consumer

1. request.timeout.ms （默认为305000ms）

1. request.timeout.ms (by default 305000ms)

配置控制客户端
等待的最长时间对于请求的响应

The configuration controls the maximum amount of time the client will wait for the response of a request

1. connections.max.idle。 ms （默认为540000ms）

1. connections.max.idle.ms (by default 540000ms)

在指定的毫秒数后关闭空闲连接
此配置。

Close idle connections after the number of milliseconds specified by this config.

这篇关于Kafka Streams错误 - 分区上的偏移提交失败，请求超时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！