如何从kafka强制删除主题 [英] how to force delete topic from kafka

问题描述

我们执行以下步骤以删除主题- hgpo.llo.prmt.processed

we do the following steps in order to delete the topic - hgpo.llo.prmt.processed

但是即使经过12小时，主题文件夹仍然没有从/var/kafka/kafka-logs

but even after 12 hours topics folders still not deleted from /var/kafka/kafka-logs

注意-我们设置-delete.topic.enable = true

note - we set - delete.topic.enable=true

  hkafka01 kafka-logs]# /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper master01:2181 --alter --topic hgpo.llo.prmt.processed--config retention.ms=1000

  WARNING: Altering topic configuration from this script has been deprecated and may be removed in future releases. Going forward, please use kafka-configs.sh for this functionality Updated config for topic "hgpo.llo.prmt.processedd"

  kafka01 kafka-logs]# /usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper master01:2181 --delete --topic hgpo.llo.prmt.processed 

  Topic hgpo.llo.prmt.processed is already marked for deletion.

如何强制删除主题?

在kafka机器(生产机器)上，我们看到:(没有可用空间)

on kafka machine ( production machine ) , we see that: ( no free space )

 /dev/sdb    15500450528 15500434144         0 100% /var/kafka

在/var/kafka/kafka-logs下，我们仍然可以看到所有主题文件夹(巨大)，例如:(down)

and under /var/kafka/kafka-logs , we still see all topic folders (huge size) as example: ( down )

因此可能仅以删除这些文件夹为例-通过

so is it possible just to remove these folders as example - by

rm -rf hgpo.llo.prmt.processed-28 ?

在/var/kafka/kafka-logs下，我们有许多主题文件夹，如:

under /var/kafka/kafka-logs we have many topic folders as :

 117G hgpo.llo.prmt.processed-28 
 117G hgpo.llo.prmt.processed-29
 117G hgpo.llo.prmt.processed-3
 117G hgpo.llo.prmt.processed-30
 117G hgpo.llo.prmt.processed-31
 117G hgpo.llo.prmt.processed-32

.

推荐答案

删除您一直尝试的方式应该可以，但是我猜Kafka在完整磁盘上不能很好地工作.

Deletion the way you've been trying should work but I'm guessing Kafka is not working well with a full disk.

由于磁盘已满，我假设Kafka不可用，可以接受短暂的停机时间.要手动删除主题，请执行以下操作:

As your disk is full, I'm assuming Kafka is not usable and a short downtime is acceptable. To delete the topic manually:

• 关闭卡夫卡
• 在所有具有 hgpo.llo.prmt.processed 副本的卡夫卡经纪人上，转到其数据目录并删除以 hgpo.llo.prmt.processed-.
• 使用 zookeeper-shell 工具连接到Zookeeper
• 运行: rmr/brokers/topics/hgpo.llo.prmt.processed
• 重新启动Kafka

• Shutdown Kafka
• On all Kafka brokers that have replicas for hgpo.llo.prmt.processed, go to their data directory and delete the folders starting with hgpo.llo.prmt.processed-.
• Connect to Zookeeper with the zookeeper-shell tool
• Run: rmr /brokers/topics/hgpo.llo.prmt.processed
• Restart Kafka

为防止这种情况再次发生，建议您查看一下Kafka提供的用于控制磁盘使用情况的保留策略

To prevent this from happening again, I suggest you take a look at the retention policies Kafka offers to control disk usage

这篇关于如何从kafka强制删除主题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！