清理策略:Compact/Delete and log.retention [英] Cleanup Policy:Compact/Delete and log.retention
问题描述
我对 Kafka 主题清理策略及其与 log.retention 的交互有疑问....
I have a question about Kafka Topic cleanup policies and their interaction of log.retention....
比如我把cleanup.policy设置为compact，compaction只会在topic的保留时间之后才开始或者保留时间对compaction没有影响?
For example, if I set cleanup.policy to compact, compaction will only start after the retention time of the topic or retention time has no effect for compaction?
问题的第二部分，如果我使用compact，delete一起，并且我有log.retention for让说1天，主题一直被压缩但是主题的内容会在一天后被删除?还是一天后实现压缩和删除?
Second part of the question, if I use compact,delete together, and I have log.retention for lets say 1 day, topic compacted all the time but content of the topic will be deleted after one day? or compaction and delete realised after one day?
感谢回答...
推荐答案
可以删除或压缩日志段，或两者兼而有之，以管理它们的大小.主题级配置 cleanup.policy 决定了主题日志段的管理方式.
Log segments can be deleted or compacted, or both, to manage their size. The topic-level configuration cleanup.policy determines the way the log segments for the topic are managed.
通过压缩清理日志
如果主题级配置 cleanup.policy 设置为 compact，则日志清理器会在后台定期压缩主题的日志.
If the topic-level configuration cleanup.policy is set to compact,the log for the topic is compacted periodically in the background by the log cleaner.
在压缩的主题中，日志只需要包含每个键的最新消息，而较早的消息可以被丢弃.
In a compacted topic,the log only needs to contain the most recent message for each key while earlier messages can be discarded.
无需将 log.retention 设置为 -1 或任何其他值.您的主题将被压缩，旧消息永远不会被删除(根据压缩规则).
There is no need to set log.retention to -1 or any other value. Your topics will be compacted and old messages never deleted (as per compaction rules).
注意只有非活动的文件段才能被压缩；活动段永远不会被压缩.
Note that only the inactive file segment can be compacted; active segment will never be compacted.
同时使用两者的日志清理
您可以同时为 cleanup.policy 配置指定 delete 和 compact 值.在这种情况下，日志被压缩，但清理过程也遵循 retention time 或 size limit 设置.
You can specify both delete and compact values for the cleanup.policy configuration at the same time. In this case, the log is compacted, but the cleanup process also follows the retention time or size limit settings.
我建议您浏览以下链接
https://ibm.github.io/event-streams/installing/容量规划/
https://kafka.apache.org/documentation/#compaction
这篇关于清理策略:Compact/Delete and log.retention的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
