Kafka流抑制会话窗口聚合 [英] Kafka Stream Suppress session-windowed-aggregation

问题描述

我已经在Kafka流应用程序中编写了以下代码:

I have written this code in a Kafka stream application:

KGroupedStream<String, foo> groupedStream = stream.groupByKey();
groupedStream.windowedBy(
SessionWindows.with(Duration.ofSeconds(3)).grace(Duration.ofSeconds(3)))
    .aggregate(() -> {...})
    .suppress(Suppressed.untilWindowCloses(unbounded()))
    .toStream()...

它应该(如果我理解正确的话)在关闭窗口后为每个Key发出记录. 不知何故，行为如下:

which should (if i understood it correctly) emit records per Key after the window is closed. Somehow the behavior is the following:

该流不会发出第一条记录，仅使用不同的Key才将其转发到第二条记录之后，然后仅在第三条记录之后发出第二条记录，依此类推.

The stream doesn't emit the first record and only forward it after the second record even with a different Key and then the second record is emitted only after the 3rd and so forth..

我尝试了多个带有"exactly_once"的StreamConfigs，并且无论是否具有缓存，这种现象仍然存在.

I have tried multiple StreamConfigs with "exactly_once" and with or without Caching also, this behavior persists.

在此先感谢您的帮助！

推荐答案

这是预期的行为.请注意，suppress()是基于事件时间的.因此，只要没有新数据到达，时间就无法提前，因此更早地收回记录是错误的，因为不能保证下一条记录可能属于当前窗口.

That is expected behavior. Note, that suppress() is based on event-time. Thus, as long as no new data arrives, time cannot advance and thus evicting the record earlier would be wrong, because there is no guarantee, that the next record might belong to the current window.

这篇关于Kafka流抑制会话窗口聚合的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！