为什么 Kafka Streams 强制为 GlobalKTable 状态存储禁用日志记录? [英] Why does Kafka Streams enforce logging disabled for GlobalKTable state stores?

问题描述

为什么全局表不能在 kafka 中启用日志记录?

Why must a global table not have logging enabled in kafka?

代码:

if (loggingEnabled) {
    throw new TopologyException("StateStore " + storeName + " for global table must not have logging enabled.");
}

当我重新启动我的 kafka 应用程序时，globalstore 中的所有数据存储都变得干净了.我怎么能让它宽容.

When I am restarting my kafka application all the datastore in globalstore is getting clean. How could I can made it tolerance.

推荐答案

GlobalKTable 使用它们的输入主题来保证容错性.因此，我不确定您的意思

GlobalKTables use their input topic to guarantee fault-tolerance. Thus, I am not sure what you mean by

当我重新启动我的 kafka 应用程序时，globalstore 中的所有数据存储都变得干净了.

When I am restarting my kafka application all the datastore in globalstore is getting clean.

只要在启用日志压缩的情况下正确配置输入主题，即使您在本地删除状态，也会在启动时重新创建 GlobalKTable 状态.

As long as the input topic is correctly configured with log-compaction enabled, the GlobalKTable state will be recreated on startup even if you delete the state locally.

启用日志记录没有意义，因为每个 KafkaStreams 实例加载整个 GlobalKTable 数据(即，数据被广播到所有实例).变更日志主题专为共享状态而设计，尽管每个分片使用一个主题分区.因此，这两个概念并不一致，而是相互矛盾.

Enabling logging does not make sense because each KafkaStreams instance loads the whole GlobalKTable data (ie, the data is broadcasted to all instances). Changelog topics are designed for shareded state though using one topic-partition per shard. Thus, both concepts don't align but contradict each other.

这篇关于为什么 Kafka Streams 强制为 GlobalKTable 状态存储禁用日志记录?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！