如何在KStream中获取偏移值 [英] How can I get the offset value in KStream

问题描述

我正在使用Kafka Streams开发PoC.现在，我需要在流使用者中获取偏移量值，并使用它为每个消息生成唯一的键(topic-offset)->hash.原因是:生产者是syslog，只有少数具有ID.我无法在使用者中生成UUID，因为在进行重新处理的情况下，我需要重新生成相同的密钥.

I'm developing a PoC with Kafka Streams. Now I need to get the offset value in the stream consumer and use it to generate an unique key (topic-offset)->hash for each message. The reason is: the producers are syslog and only few of them have ID's. I cannot generate an UUID in the consumer because in case of reprocess I need to regenerate the same key.

我的问题是:org.apache.kafka.streams.processor.ProcessorContext类公开了一个返回值的.offset()方法，但是我使用的是KStream而不是Processor，所以我找不到返回相同结果的方法.

My problem is: the org.apache.kafka.streams.processor.ProcessorContext class expose an .offset() method that returns the value, but I'm using KStream instead of the Processor, and I couldn't find a method that returns the same thing.

任何人都知道如何从Kstream中提取每一行的消费者价值吗? 预先感谢

Anybody knows how to extract the consumer value for each row from a Kstream? thanks in advance

推荐答案

您可以通过process(...)，transform(...)和transformValues(...)使用混合匹配DSL和Processor API.

You can use mix-and-match DSL and Processor API via process(...), transform(...), and transformValues(...).

它使您可以像使用普通处理器API一样访问当前记录的偏移量.您似乎想使用KStream#transform(...).

It allows you to access the current record offset similar to plain Processor API. In you case, it seems you want to use KStream#transform(...).

这篇关于如何在KStream中获取偏移值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！