Kafka 不会删除主题中的旧消息 [英] Kafka doesn't delete old messages in topics

问题描述

在 kafka 中，我在 server.properties

In kafka I have set retention policy to 3 days in server.properties

############################# Log Retention Policy #############################
...
log.retention.hours=72
...

主题已将 retention.ms 设置为 172800000(48 小时).

Topics has retention.ms set to 172800000 (48h).

然而，文件夹/tmp/kafka-logs中仍然有旧数据，并且没有被删除.更改这些属性后我等了几个小时.

However, there are still old data in the folder /tmp/kafka-logs and none are being deleted. I waited few hours after changing those properties.

有什么需要设置的吗?当前所有主题都在生产和消费中.

Is there something that needs to be set? All topics are being produced to and consumed from currently.

推荐答案

Edit: 根据 kafka 文档，cleanup.policy 似乎应该默认为 delete.

it seems the cleanup.policy should default to delete according to kafka documentation.

retention.ms 或 retention.bytes 指定删除数据的方式.

retention.ms or retention.bytes specify the way the data are deleted.

关键是将log.cleanup.policy 设置为compact 或delete.我没有设置这个.

The key was to set log.cleanup.policy to compact or delete. I had not set this.

运行:kafka-topics --zookeeper 127.0.0.1:2181 --topic topic1 --describe 显示在主题上设置的属性，例如.Configs:retention.ms=172800000,cleanup.policy=compact

Running: kafka-topics --zookeeper 127.0.0.1:2181 --topic topic1 --describe shows properties set on topic, eg. Configs:retention.ms=172800000,cleanup.policy=compact

必须设置 cleanup.policy.另外我手动设置 retention.ms/retention.bytes 来控制清理触发器.

The cleanup.policy has to be set. Also I manually set retention.ms / retention.bytes to control cleanup trigger.

这篇关于Kafka 不会删除主题中的旧消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！