kafka.common.OffsetOutOfRangeException的含义 [英] What kafka.common.OffsetOutOfRangeException means

问题描述

我正在尝试通过Apache Kafka加载数据并不断出现此错误:

I'm trying to load data through Apache Kafka and constantly getting this error:

kafka.common.OffsetOutOfRangeException:偏移量1003786超出范围在kafka.log.Log $ .findRange(Log.scala:46)在kafka.log.Log.read(Log.scala:264)在kafka.server.KafkaRequestHandlers.kafka $ server $ KafkaRequestHandlers $$ readMessageSet(KafkaRequestHandlers.scala:112)在kafka.server.KafkaRequestHandlers $$ anonfun $ 2.apply(KafkaRequestHandlers.scala:101)在kafka.server.KafkaRequestHandlers $$ anonfun $ 2.apply(KafkaRequestHandlers.scala:100)在scala.collection.TraversableLike $$ anonfun $ map $ 1.apply(TraversableLike.scala:206)在scala.collection.TraversableLike $$ anonfun $ map $ 1.apply(TraversableLike.scala:206)在scala.collection.IndexedSeqOptimized $ class.foreach(IndexedSeqOptimized.scala:34)在scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34)在scala.collection.TraversableLike $ class.map(TraversableLike.scala:206)在scala.collection.mutable.ArrayOps.map(ArrayOps.scala:34)在kafka.server.KafkaRequestHandlers.handleMultiFetchRequest(KafkaRequestHandlers.scala:100)在kafka.server.KafkaRequestHandlers $$ anonfun $ handlerFor $ 3.apply(KafkaRequestHandlers.scala:40)在kafka.server.KafkaRequestHandlers $$ anonfun $ handlerFor $ 3.apply(KafkaRequestHandlers.scala:40)在kafka.network.Processor.handle(SocketServer.scala:296)在kafka.network.Processor.read(SocketServer.scala:319)在kafka.network.Processor.run(SocketServer.scala:214)在java.lang.Thread.run(Thread.java:724)

kafka.common.OffsetOutOfRangeException: offset 1003786 is out of range at kafka.log.Log$.findRange(Log.scala:46) at kafka.log.Log.read(Log.scala:264) at kafka.server.KafkaRequestHandlers.kafka$server$KafkaRequestHandlers$$readMessageSet(KafkaRequestHandlers.scala:112) at kafka.server.KafkaRequestHandlers$$anonfun$2.apply(KafkaRequestHandlers.scala:101) at kafka.server.KafkaRequestHandlers$$anonfun$2.apply(KafkaRequestHandlers.scala:100) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34) at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34) at scala.collection.TraversableLike$class.map(TraversableLike.scala:206) at scala.collection.mutable.ArrayOps.map(ArrayOps.scala:34) at kafka.server.KafkaRequestHandlers.handleMultiFetchRequest(KafkaRequestHandlers.scala:100) at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$3.apply(KafkaRequestHandlers.scala:40) at kafka.server.KafkaRequestHandlers$$anonfun$handlerFor$3.apply(KafkaRequestHandlers.scala:40) at kafka.network.Processor.handle(SocketServer.scala:296) at kafka.network.Processor.read(SocketServer.scala:319) at kafka.network.Processor.run(SocketServer.scala:214) at java.lang.Thread.run(Thread.java:724)

此异常是什么意思，我该如何解决?

What does this exception mean and how I can fix it?

推荐答案

OffsetOutOfRangeException 通常指示客户端请求了服务器上不再可用的范围.
由于您的Kafka设置中的保留政策，主题日志已不存在，因此可能会发生这种情况.
如果您使用的是 SimpleConsumer ，则需要在代码中处理OffsetOutOfRange异常.理想情况下，您的使用者应该发出OffsetRequest以获得服务器上当前可用的最新/最早的偏移量，然后在FetchRequest中使用它(作为参数)

OffsetOutOfRangeException generally indicates that client has requested a range no longer available on the server.
This could happen as the topic log does not exists anymore based on the retention policy in your Kafka setup.
if you are using SimpleConsumer you will need to handle the OffsetOutOfRange Exception in your code. Ideally your consumer should issue an OffsetRequest to get the latest/earliest offset currently available on the server, and then use this in your FetchRequest (as a parameter)

这篇关于kafka.common.OffsetOutOfRangeException的含义的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！