为什么Kafka Streams强制禁用GlobalKTable状态存储的日志记录? [英] Why does Kafka Streams enforce logging disabled for GlobalKTable state stores?

问题描述

为什么必须在kafka中未启用全局表的日志记录?

Why must a global table not have logging enabled in kafka?

代码:

if (loggingEnabled) {
    throw new TopologyException("StateStore " + storeName + " for global table must not have logging enabled.");
}

当我重新启动我的kafka应用程序时，globalstore中的所有数据存储都变得干净了.我怎么能忍受呢?

When I am restarting my kafka application all the datastore in globalstore is getting clean. How could I can made it tolerance.

推荐答案

GlobalKTable 使用其输入主题来保证容错能力.因此，我不确定您的意思

GlobalKTables use their input topic to guarantee fault-tolerance. Thus, I am not sure what you mean by

当我重启我的kafka应用程序时，globalstore中的所有数据存储都变得干净了.

When I am restarting my kafka application all the datastore in globalstore is getting clean.

只要正确配置了输入主题并启用了日志压缩，即使您在本地删除状态，也会在启动时重新创建 GlobalKTable 状态.

As long as the input topic is correctly configured with log-compaction enabled, the GlobalKTable state will be recreated on startup even if you delete the state locally.

启用日志记录没有意义，因为每个 KafkaStreams 实例都加载整个 GlobalKTable 数据(即，该数据将广播到所有实例).尽管每个分片使用一个主题分区，但Changelog主题专为 shared 状态而设计.因此，这两个概念并不一致，却相互矛盾.

Enabling logging does not make sense because each KafkaStreams instance loads the whole GlobalKTable data (ie, the data is broadcasted to all instances). Changelog topics are designed for shareded state though using one topic-partition per shard. Thus, both concepts don't align but contradict each other.

这篇关于为什么Kafka Streams强制禁用GlobalKTable状态存储的日志记录?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！