kafka.common.OffsetOutOfRangeException 是什么意思 [英] What kafka.common.OffsetOutOfRangeException means

问题描述

我正在尝试通过 Apache Kafka 加载数据并不断收到此错误:

I'm trying to load data through Apache Kafka and constantly getting this error:

kafka.common.OffsetOutOfRangeException:偏移量 1003786 超出范围在 kafka.log.Log$.findRange(Log.scala:46)在 kafka.log.Log.read(Log.scala:264)在 kafka.server.KafkaRequestHandlers.kafka$server$KafkaRequestHandlers$$readMessageSet(KafkaRequestHandlers.scala:112)在 kafka.server.KafkaRequestHandlers$$anonfun$2.apply(KafkaRequestHandlers.scala:101)在 kafka.server.KafkaRequestHandlers$$anonfun$2.apply(KafkaRequestHandlers.scala:100)在 scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)在 scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)在 scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)在 scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34)在 scala.collection.TraversableLike$class.map(TraversableLike.scala:206)在 scala.collection.mutable.ArrayOps.map(ArrayOps.scala:34)在 kafka.server.KafkaRequestHandlers.handleMultiFetchRequest(KafkaRequestHandlers.scala:100)在 kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$3.apply(KafkaRequestHandlers.scala:40)在 kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$3.apply(KafkaRequestHandlers.scala:40)在 kafka.network.Processor.handle(SocketServer.scala:296)在 kafka.network.Processor.read(SocketServer.scala:319)在 kafka.network.Processor.run(SocketServer.scala:214)在 java.lang.Thread.run(Thread.java:724)

kafka.common.OffsetOutOfRangeException: offset 1003786 is out of range at kafka.log.Log$.findRange(Log.scala:46) at kafka.log.Log.read(Log.scala:264) at kafka.server.KafkaRequestHandlers.kafka$server$KafkaRequestHandlers$$readMessageSet(KafkaRequestHandlers.scala:112) at kafka.server.KafkaRequestHandlers$$anonfun$2.apply(KafkaRequestHandlers.scala:101) at kafka.server.KafkaRequestHandlers$$anonfun$2.apply(KafkaRequestHandlers.scala:100) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34) at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34) at scala.collection.TraversableLike$class.map(TraversableLike.scala:206) at scala.collection.mutable.ArrayOps.map(ArrayOps.scala:34) at kafka.server.KafkaRequestHandlers.handleMultiFetchRequest(KafkaRequestHandlers.scala:100) at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$3.apply(KafkaRequestHandlers.scala:40) at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$3.apply(KafkaRequestHandlers.scala:40) at kafka.network.Processor.handle(SocketServer.scala:296) at kafka.network.Processor.read(SocketServer.scala:319) at kafka.network.Processor.run(SocketServer.scala:214) at java.lang.Thread.run(Thread.java:724)

这个异常是什么意思，我该如何解决?

What does this exception mean and how I can fix it?

推荐答案

OffsetOutOfRangeException 通常表示客户端请求了服务器上不再可用的范围.
根据 Kafka 设置中的保留策略，主题日志不再存在，因此可能会发生这种情况.
如果您使用 SimpleConsumer，则需要处理代码中的 OffsetOutOfRange 异常.理想情况下，您的消费者应该发出 OffsetRequest 以获取服务器上当前可用的最新/最早偏移量，然后在您的 FetchRequest 中使用它(作为参数)

OffsetOutOfRangeException generally indicates that client has requested a range no longer available on the server.
This could happen as the topic log does not exists anymore based on the retention policy in your Kafka setup.
if you are using SimpleConsumer you will need to handle the OffsetOutOfRange Exception in your code. Ideally your consumer should issue an OffsetRequest to get the latest/earliest offset currently available on the server, and then use this in your FetchRequest (as a parameter)

这篇关于kafka.common.OffsetOutOfRangeException 是什么意思的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！