Kafka:如何使用 Java API 从主题中删除记录? [英] Kafka: How to delete records from a topic using Java API?

问题描述

我正在寻找一种从 Kafka 主题中删除(完全删除)消耗记录的方法.我知道有几种方法可以做到这一点，例如，通过更改主题的保留时间或删除 Kafka-logs 文件夹.但我正在寻找的是一种使用 Java API 删除某个主题的一定数量记录的方法，如果可能的话.

I'm looking for a way to delete (completely remove) consumed records from a Kafka topic. I know there are several ways of doing this, by either changing the retention time for the topic or removing the Kafka-logs folder, for example. But what I'm looking for is a way to delete a certain amount of records for a topic using the Java API, if that is possible.

我尝试过测试 AdminClient API，特别是 adminclient.deleteRecords(recordsToDelete) 方法.但是如果我没记错的话，那个方法只是改变了主题中的偏移量，实际上并没有从硬盘驱动器中删除所说的记录.

I've tried testing the AdminClient API, specifically the adminclient.deleteRecords(recordsToDelete) method. But if I'm not mistaken, that method only Changes the offsets in the topic, not actually deleting said records from the hard drive.

是否有真正从硬盘驱动器中删除记录的 Java API?

Is there a Java API that does actually remove the records from the hard drive?

推荐答案

Kafka 不支持从主题中删除记录.它的工作方式是构建一个消息缓冲区，该缓冲区随着消息推送到它而增长.而读取消息的客户端基本上只持有该缓冲区的偏移量.因此 Kafka 中的客户端基本上处于只读"模式并且无法更改缓冲区.考虑一个案例，当几个不同的客户端(不同的客户端组)读取相同的主题并且每个都保存自己的偏移量时.如果有人开始从设置了偏移量的缓冲区中删除消息，会发生什么.

Kafka doesn't support removing records from topics. The way it works is by building a buffer of messages that grow as message pushed to it. While the client that reads the messages basically only holding an offset to that buffer. So clients in Kafka are basically in "read-only" mode and can't alter the buffer. Think about a case when several different clients (different client-groups) reading the same topic and each saves its own offset. what would happen if someone will start deleting messages from the buffer where the offset is set to.

这篇关于Kafka:如何使用 Java API 从主题中删除记录?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！