如何清理 Kafka 0.8.2 中压缩日志中的旧段 [英] How clean old segments from compacted log in Kafka 0.8.2

问题描述

我知道在新的 Kafka 版本中，我们有新的保留策略选项 - 日志压缩，删除具有相同密钥的旧版本消息.但是很长一段时间后，我们会得到太多带有旧消息的压缩日志段.我们如何自动清理这个压缩日志?

I know that in new Kafka versions we have new retention policy option - compaction of log which delete old version of messages with same keys. But after long time we will get too many compacted log segments with old messages. How can we clean this compacted log automatically?

UDPATE:

我应该澄清一下，当时我们需要紧凑的日志和清理旧消息的方法.我在这里找到了关于相同问题的讨论 http://grokbase.com/t/kafka/users/14bv6gaz0t/kafka-0-8-2-log-cleaner但没有找到我们如何手动发出消息的汤姆斯通标记并且不知道这一点.

I should clarify that we need compact log and way to clean up old messages this in those time. I found discussion for same problem here http://grokbase.com/t/kafka/users/14bv6gaz0t/kafka-0-8-2-log-cleaner but not found how we can manually issue thomstone markers for message and have not any idea this.

推荐答案

这个问题已经很老了，但我想我会就此事提供最新的更新.有一个功能(https://issues.apache.org/jira/browse/KAFKA-4015) 已解决并计划在 0.10.1.0 版本中发布.

This question is quite old, but I thought I'd give the latest update on the matter. There is a feature (https://issues.apache.org/jira/browse/KAFKA-4015) which is already resolved and is scheduled for the 0.10.1.0 release.

这篇关于如何清理 Kafka 0.8.2 中压缩日志中的旧段的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！