如果我在 Kafka 中有事务性生产者，我可以使用 Kafka Streams 读取一次消息吗? [英] If I have Transactional Producer in Kafka can I read exactly once messages with Kafka Streams?

问题描述

我想拥有 Exactly-once 语义，但我不想使用 Consumer 读取消息.我宁愿使用 Kafka Streams AP 阅读消息.如果我将 processing.guarantee=exactly_once 添加到 Stream 配置中，是否会保留恰好一次语义?

I would like to have Exactly-once semantics, but I don't want to read message with Consumer. I'd rather read messages with Kafka Streams AP. If I add processing.guarantee=exactly_once to Stream configuration, will exactly-once semantics be kept?

推荐答案

Exactly-once 处理基于读-处理-写模式.Kafka Streams 使用这种模式，因此，如果您编写一个将结果写回 Kafka 主题的常规 Kafka Streams 应用程序，您将获得一次处理保证.

Exactly-once processing is based on a read-process-write pattern. Kafka Streams uses this pattern and thus, if you write a regular Kafka Streams application that writes the result back to a Kafka topic, you will get exactly-once processing guarantees.

请注意，此保证不涵盖副作用.另请注意，如果失败，可能会在内部进行重试.Exactly-once 意味着您在输出主题中看到的结果与没有发生错误(因此不会重试)一样.

Note, that side-effects are not covered by this guarantee. Also note, that there might be retries internally in case of failure. Exactly-once implies, that the result you see in the output topic, is the same as if no error (and thus not retries) would have happened.

有关更多详细信息，您可能想观看有关 Kafka 中完全一次保证的谈话录音.Confluent 网页上有多个可用的(免责声明:我是 Confluent 的一名员工):https://www.confluent.io/resources/

For more details, you might want to watch a talk recording about exactly-once guarantees in Kafka. There are multiple available at Confluent webpage (disclaimer: I am an employee at Confluent): https://www.confluent.io/resources/

这篇关于如果我在 Kafka 中有事务性生产者，我可以使用 Kafka Streams 读取一次消息吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！