Kafka 不会删除主题中的旧消息 [英] Kafka doesn't delete old messages in topics
问题描述
在 kafka 中,我在 server.properties
In kafka I have set retention policy to 3 days in server.properties
############################# Log Retention Policy #############################
...
log.retention.hours=72
...
主题已将 retention.ms
设置为 172800000
(48 小时).
Topics has retention.ms
set to 172800000
(48h).
然而,文件夹/tmp/kafka-logs中仍然有旧数据,并且没有被删除.更改这些属性后我等了几个小时.
However, there are still old data in the folder /tmp/kafka-logs and none are being deleted. I waited few hours after changing those properties.
有什么需要设置的吗?当前所有主题都在生产和消费中.
Is there something that needs to be set? All topics are being produced to and consumed from currently.
推荐答案
Edit: 根据 kafka 文档,cleanup.policy
似乎应该默认为 delete
.
it seems the cleanup.policy
should default to delete
according to kafka documentation.
retention.ms
或 retention.bytes
指定删除数据的方式.
retention.ms
or retention.bytes
specify the way the data are deleted.
关键是将log.cleanup.policy
设置为compact
或delete
.我没有设置这个.
The key was to set log.cleanup.policy
to compact
or delete
. I had not set this.
运行:kafka-topics --zookeeper 127.0.0.1:2181 --topic topic1 --describe
显示在主题上设置的属性,例如.Configs:retention.ms=172800000,cleanup.policy=compact
Running: kafka-topics --zookeeper 127.0.0.1:2181 --topic topic1 --describe
shows properties set on topic, eg. Configs:retention.ms=172800000,cleanup.policy=compact
必须设置 cleanup.policy
.另外我手动设置 retention.ms
/retention.bytes
来控制清理触发器.
The cleanup.policy
has to be set. Also I manually set retention.ms
/ retention.bytes
to control cleanup trigger.
这篇关于Kafka 不会删除主题中的旧消息的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!