是否可以将Kafka Streams配置为等待KTable加载? [英] Can Kafka Streams be configured to wait for KTable to load?
问题描述
我正在使用物化的KTable来与我的KStream进行左连接(而流是在左侧).
I'm using materialized KTable to use for left join with my KStream(while the stream is the left side).
但是，它似乎可以立即处理，而无需等待KTable的当前版本加载..
However, it seem to process immediately, without waiting for the current version of the KTable to load..
我在KTable的源主题中有很多值，并且当我启动应用程序时，很多联接失败了(嗯，不是真的，因为它是左联接).
I have a lot of values in my source topic for the KTable and when I start the application, a lot of joins fail(well, not really since it is a left join).
我可以延迟启动，以便等待初始主题加载吗?
Can I make it start in delay so it would wait for the initial topic load?
推荐答案
在Kafka Streams中处理是时间同步的.因此，基于记录时间戳顺序处理表输入主题和流输入主题.从语义上讲，这是合理的，因为在流表连接中，您不想将流记录与旧版本或KTable的较新版本连接在一起，而是希望基于流记录时间戳记使用正确的版本进行连接.
Processing is time synchronized in Kafka Streams. Hence, the table input topic and stream input topic are processed based on record timestamp order. This is semantically sound, because on a stream-table join, you don't want to join a stream record with an older version nor with a newer version of the KTable, but with the right version based on the stream record timestamp.
如果您的数据未正确加上时间戳，则可以尝试通过builder.table(..., Consumed.with(...))指定一个自定义时间戳提取器，以返回确保正确行为的时间戳(即，是否小于第一个流记录的时间戳?)
If your data is not properly timestamped, you can try to specify a custom timestamp extractor for via builder.table(..., Consumed.with(...)) to return timestamps that ensure proper behavior (ie, maybe smaller than timestamp of the first stream record?)
请注意，正确的时间戳同步需要Kafka Streams 2.1.旧版本仅以尽力而为的方式同步时间，可能无法提供您想要的行为.有关更多详细信息，请参阅KIP-353.
Note, that a proper timestamp synchronization requires Kafka Streams 2.1. Older version synchronize time in best effort manner only and may not provide the behavior you want. For more details, see KIP-353.
这篇关于是否可以将Kafka Streams配置为等待KTable加载?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
