Kafka Streams - 处理器上下文提交 [英] Kafka Streams - Processor context commit

问题描述

我们应该自己在 Processor 实现中调用 processorContext.commit() 吗?我的意思是在预定的 Punctuator 实现或 process 方法内调用 commit 方法.

should we ever invoke processorContext.commit() in Processor implementation by ourselves? I mean invoking commit method inside scheduled Punctuator implementation or inside process method.

我们应该在哪些用例中这样做，我们是否需要这样做?该问题与带有 transform() 和 Processor API 的 Kafka DSL 相关.

in which use cases should we do that, and do we need that at all? the question relates to both Kafka DSL with transform() and Processor API.

似乎 Kafka Streams 自己处理它，也调用 processorContext.commit() 并不能保证它会立即完成.

seems Kafka Streams handles it by itself, also invoking processorContext.commit() does not guarantee that it will be done immediately.

推荐答案

可以从处理器或从标点符号调用 commit() -- 这就是提供此 API 的原因.

It is ok to call commit() -- either from the Processor or from a Punctuation -- that's why this API is offered.

虽然 Kafka Streams 定期(可配置)提交，但您可以在使用时请求中间提交.一个示例用例是，您通常会进行廉价的计算，但有时您会做一些昂贵的事情并希望在此操作后尽快提交而不是等待下一次提交间隔(以减少在昂贵的操作和下一个提交间隔).另一个用例是，如果您将提交间隔设置为 MAX_VALUE，这将有效地禁用"常规提交并根据您的业务逻辑决定何时提交.

While Kafka Streams commits on a regular (configurable) interval, you can request intermediate commits when you use it. One example use case would be, that you usually do cheap computation, but sometimes you do something expensive and want to commit asap after this operation instead of waiting for the next commit interval (to reduce the likelihood of a failure after the expensive operation and the next commit interval). Another use case would be, if you set the commit interval to MAX_VALUE what effectively "disables" regular commits and to decide when to commit base on your business logic.

我想，对于大多数用例来说，调用 commit() 并不是必需的.

I guess, that calling commit() is not necessary for most use cases thought.

这篇关于Kafka Streams - 处理器上下文提交的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！