如何从Kafka 0.8.2中的压缩日志中清除旧段 [英] How clean old segments from compacted log in Kafka 0.8.2

问题描述

我知道在新的Kafka版本中，我们有新的保留策略选项-压缩日志，删除具有相同密钥的旧版本的邮件.但是经过很长一段时间，我们将获得太多带有旧消息的压缩日志段.我们如何自动清除此压缩日志?

I know that in new Kafka versions we have new retention policy option - compaction of log which delete old version of messages with same keys. But after long time we will get too many compacted log segments with old messages. How can we clean this compacted log automatically?

UDPATE :

我应该澄清一下，我们需要紧凑的日志和方法来清理那些旧的消息. 我在这里找到了针对同一问题的讨论 http://grokbase.com/t/kafka/users/14bv6gaz0t/kafka-0-8-2-log-cleaner 但找不到如何手动发布消息的墓碑标记，对此一无所知.

I should clarify that we need compact log and way to clean up old messages this in those time. I found discussion for same problem here http://grokbase.com/t/kafka/users/14bv6gaz0t/kafka-0-8-2-log-cleaner but not found how we can manually issue thomstone markers for message and have not any idea this.

推荐答案

这个问题已经很老了，但是我想我会提供最新的信息.有一个功能( https://issues.apache.org/jira/browse/KAFKA- 4015 )，该问题已经解决，并计划在0.10.1.0版本中发布.

This question is quite old, but I thought I'd give the latest update on the matter. There is a feature (https://issues.apache.org/jira/browse/KAFKA-4015) which is already resolved and is scheduled for the 0.10.1.0 release.

这篇关于如何从Kafka 0.8.2中的压缩日志中清除旧段的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！