Kafka:如何使用Java API从主题中删除记录? [英] Kafka: How to delete records from a topic using Java API?

问题描述

我正在寻找一种从Kafka主题中删除(完全删除)消耗的记录的方法.我知道有几种方法可以做到这一点，例如，可以更改主题的保留时间或删除Kafka-logs文件夹.但是，我正在寻找的是一种使用Java API删除某个主题的一定数量的记录的方法，如果可能的话.

I'm looking for a way to delete (completely remove) consumed records from a Kafka topic. I know there are several ways of doing this, by either changing the retention time for the topic or removing the Kafka-logs folder, for example. But what I'm looking for is a way to delete a certain amount of records for a topic using the Java API, if that is possible.

我尝试测试AdminClient API，特别是 adminclient.deleteRecords(recordsToDelete)方法.但是，如果我没记错的话，该方法只会更改主题中的偏移量，而实际上不会从硬盘驱动器中删除所述记录.

I've tried testing the AdminClient API, specifically the adminclient.deleteRecords(recordsToDelete) method. But if I'm not mistaken, that method only Changes the offsets in the topic, not actually deleting said records from the hard drive.

是否有Java API确实从硬盘驱动器中删除了记录?

Is there a Java API that does actually remove the records from the hard drive?

推荐答案

Kafka不支持从主题中删除记录.它的工作方式是建立一个消息缓冲区，该消息缓冲区随着消息被推送到消息缓冲区而增长.而读取消息的客户端基本上只保留该缓冲区的偏移量.因此，Kafka中的客户端基本上处于只读"模式，无法更改缓冲区.考虑几个不同的客户端(不同的客户端组)读取相同的主题并各自保存自己的偏移量的情况.如果有人开始从偏移量设置为的缓冲区中删除邮件，将会发生什么情况.

Kafka doesn't support removing records from topics. The way it works is by building a buffer of messages that grow as message pushed to it. While the client that reads the messages basically only holding an offset to that buffer. So clients in Kafka are basically in "read-only" mode and can't alter the buffer. Think about a case when several different clients (different client-groups) reading the same topic and each saves its own offset. what would happen if someone will start deleting messages from the buffer where the offset is set to.

这篇关于Kafka:如何使用Java API从主题中删除记录?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！