Kafka - 未压缩的主题与压缩的主题 [英] Kafka - uncompacted topics Vs compacted topics

问题描述

我在掌握 Kafka Streams 和 ksqlDB"一书中遇到了以下两个短语:和作者使用了两个术语，它们的真正含义是压缩主题"吗?和未压缩的主题"

他们对日志压缩"有什么了解吗??

<块引用>

表可以被认为是对数据库的更新.在此日志视图中，仅保留每个键的当前状态(给定键的最新记录或某种聚合).表格通常由压缩主题构建而成.

<块引用>

流可以被认为是数据库术语中的插入.每个不同的记录都保留在日志的这个视图中.流通常由未压缩的主题构建.

解决方案

是的，log compaction 根据 kafka docs

<块引用>

日志压缩确保 Kafka 始终至少为单个主题分区的数据日志中的每个消息键保留最后一个已知值

https://kafka.apache.org/documentation/#compaction

如果在主题上启用了日志压缩，当分区日志中有具有相同键的更新版本时，Kafka 会删除所有旧记录.

有关日志压缩的更详细说明，请参阅 - https://medium.com/swlh/introduction-to-topic-log-compaction-in-apache-kafka-3e4d4afd2262

I come across the following two phrases from the book "Mastering Kafka Streams and ksqlDB" and author used two terms, what does they really mean "compacted topics" and "uncompacted topics"

Does they got anything to with respect to "log compaction" ?

Tables can be thought of as updates to a database. In this view of the logs, only the current state (either the latest record for a given key or some kind of aggregation) for each key is retained. Tables are usually built from compacted topics.

Streams can be thought of as inserts in database parlance. Each distinct record remains in this view of the log. Streams are usually built from uncompacted topics.

解决方案

Yes, log compaction according to kafka docs

Log compaction ensures that Kafka will always retain at least the last known value for each message key within the log of data for a single topic partition

https://kafka.apache.org/documentation/#compaction

If log compaction is enabled on topic, Kafka removes any old records when there is a newer version of it with the same key in the partition log.

For more detailed explanation of log compaction refer - https://medium.com/swlh/introduction-to-topic-log-compaction-in-apache-kafka-3e4d4afd2262

这篇关于Kafka - 未压缩的主题与压缩的主题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！