Kafka Stream 抑制会话窗口聚合 [英] Kafka Stream Suppress session-windowed-aggregation

问题描述

我在 Kafka 流应用程序中编写了此代码:

I have written this code in a Kafka stream application:

KGroupedStream<String, foo> groupedStream = stream.groupByKey();
groupedStream.windowedBy(
SessionWindows.with(Duration.ofSeconds(3)).grace(Duration.ofSeconds(3)))
    .aggregate(() -> {...})
    .suppress(Suppressed.untilWindowCloses(unbounded()))
    .toStream()...

应该(如果我理解正确的话)在窗口关闭后为每个键发出记录.不知何故，行为如下:

which should (if i understood it correctly) emit records per Key after the window is closed. Somehow the behavior is the following:

该流不会发出第一条记录，即使使用不同的密钥，也只会在第二条记录之后转发它，然后仅在第三条之后才发出第二条记录，依此类推.

The stream doesn't emit the first record and only forward it after the second record even with a different Key and then the second record is emitted only after the 3rd and so forth..

我已经尝试了多个带有exactly_once"的 StreamConfig，无论是否使用缓存，这种行为仍然存在.

I have tried multiple StreamConfigs with "exactly_once" and with or without Caching also, this behavior persists.

预先感谢您的帮助！

推荐答案

这是预期的行为.请注意， suppress() 基于事件时间.因此，只要没有新数据到达，时间就不能前进，因此提前驱逐记录是错误的，因为不能保证下一条记录可能属于当前窗口.

That is expected behavior. Note, that suppress() is based on event-time. Thus, as long as no new data arrives, time cannot advance and thus evicting the record earlier would be wrong, because there is no guarantee, that the next record might belong to the current window.

这篇关于Kafka Stream 抑制会话窗口聚合的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！