如何在不删除主题的情况下删除/清除Kafka排队的消息 [英] How do I delete/clean Kafka queued messages without deleting Topic
问题描述
是否可以在不删除Kafka主题的情况下删除队列消息?
我想在激活使用者时删除队列消息.
我知道有几种类似的方法:
-
重置保留时间
$ ./bin/kafka-topics.sh --zookeeper本地主机:2181 --alter --topic MyTopic --configtention.ms = 1000
-
删除kafka文件
$ rm -rf/data/kafka-logs/< topic/Partition_name>
在0.11或更高版本中,您可以运行 bin/kafka-delete-records.sh
命令将邮件标记为删除.>
https://github.com/apache/kafka/blob/trunk/bin/kafka-delete-records.sh
例如,发布100条消息
seq 100 |./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic mytest
然后使用新的 kafka-delete-records.sh
删除这100条消息中的90条命令行工具
<代码> ./bin/kafka-delete-records.sh --bootstrap-server localhost:9092 --offset-json-file ./offsetfile.json
其中 offsetfile.json
包含
<代码> {"partitions":[{"topic":"mytest","partition":0,"offset":90}],"version":1}
,然后从头开始使用这些消息,以验证100条消息中的90条确实被标记为已删除.
<代码> ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mytest --from-beginning919293949596979899100
Is there any way to delete queue messages without deleting Kafka topics?
I want to delete queue messages when activating the consumer.
I know there are several ways like:
Resetting retention time
$ ./bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic MyTopic --config retention.ms=1000
Deleting kafka files
$ rm -rf /data/kafka-logs/<topic/Partition_name>
In 0.11 or higher you can run the bin/kafka-delete-records.sh
command to mark messages for deletion.
https://github.com/apache/kafka/blob/trunk/bin/kafka-delete-records.sh
For example, publish 100 messages
seq 100 | ./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic mytest
then delete 90 of those 100 messages with the new kafka-delete-records.sh
command line tool
./bin/kafka-delete-records.sh --bootstrap-server localhost:9092 --offset-json-file ./offsetfile.json
where offsetfile.json
contains
{"partitions": [{"topic": "mytest", "partition": 0, "offset": 90}], "version":1 }
and then consume the messages from the beginning to verify that 90 of the 100 messages are indeed marked as deleted.
./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mytest --from-beginning
91
92
93
94
95
96
97
98
99
100
这篇关于如何在不删除主题的情况下删除/清除Kafka排队的消息的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!