以KakfaIO从给定的偏移量开始 [英] start at a given offset with KakfaIO

问题描述

我正在使用KafkaIO.read()，我想从特定的偏移量开始消耗.

I'm using KafkaIO.read() and I'd like to start consuming from a specific offset.

在某些时候，曾经有一种KafkaIO.read().withStartFromCheckpointMark()方法来做到这一点.

At some point there used to be a KafkaIO.read().withStartFromCheckpointMark() method to do that.

我从文档可以通过以下方式实现:

I see from the documentation that there is a way via:

跑步者提供的KafkaCheckpointMark；

KafkaCheckpointMark provided by runner;

我该怎么做?

谢谢

推荐答案

没有直接支持，但有几个选项:

There is no direct support, but there are couple of options:

• withStartReadTime()可能更适合.
• 您可以创建一个group.id并在该组中提交偏移量.在KafkaConsumer配置中设置group.id时，KafkaIO默认为从为该组提交的偏移量恢复.您也可以选择在KafkaConsumer中启用auto.commit.请参阅JavaDoc中的高级Kafka配置".

• withStartReadTime() might be better suited.
• You can create a group.id and commit offsets in that group. When you set group.id in KafkaConsumer config, KafkaIO defaults to resuming from offsets committed for the group. You can also chose to enable auto.commit in KafkaConsumer. See 'Advanced Kafka Configuration' in JavaDoc.

这篇关于以KakfaIO从给定的偏移量开始的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！