如何在kafka机器重启后仍然保留kafka保留字节和kafka保留段 [英] how to remaine the kafka retentions-bytes and kafka retention-segment even after kafka machine reboot

问题描述

我们为主题- topic_test

[root@confluent01 ~]# kafka-topics --zookeeper localhost:2181 --alter --topic  topic_test --config retention.bytes=104857600
WARNING: Altering topic configuration from this script has been deprecated and may be removed in future releases.
         Going forward, please use kafka-configs.sh for this functionality
Updated config for topic "topic_test".

现在，我们从Zookeeper中验证保留字节:

Now we verify the retention bytes from the zookeeper:

[root@confluent01 ~]#  zookeeper-shell confluent01:2181 get /config/topics/topic_test
Connecting to confluent1:2181
{"version":1,"config":{"retention.bytes":"104857600"}}
cZxid = 0xb30a00000038

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
ctime = Mon Jun 29 11:42:30 GMT 2020
mZxid = 0xb31100008978
mtime = Wed Jul 22 19:22:20 GMT 2020
pZxid = 0xb30a00000038
cversion = 0
dataVersion = 7
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 54
numChildren = 0

现在我们对kafka confluent01计算机执行了 reboot

now we performed reboot to the kafka confluent01 machine

机器启动且kafka服务成功启动后，我们再次检查了zookeeper的保留字节:

after machines started and kafka service start successfully , we checked again the retention-bytes from zookeeper:

但是现在(在计算机重新启动后)，我们可以看到在Zookeeper中未配置保留字节

but now ( after machine reboot ) we can see that the retention bytes isn't configured in zookeeper

[root@confluent01 ~]#zookeeper-shell confluent01:2181 get /config/topics/topic_test
Connecting to confluent1:2181

WATCHER::

WatchedEvent state:SyncConnected type:None path:null  no retention bytes value 

{"version":1,"config":{}}
cZxid = 0xb30a00000038
ctime = Mon Jun 29 11:42:30 GMT 2020
mZxid = 0xb3110000779b
mtime = Wed Jul 22 14:09:19 GMT 2020
pZxid = 0xb30a00000038
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 25
numChildren = 0

问题是-即使重启kafka机器后，如何仍保留保留字节?

the question is - how to remain the retention bytes even after restart of kafka machine ?

注意-我们不想使用 server.properties 中的保留字节因为我们为每个主题设置了不同的保留字节

NOTE - we not want to use the retention bytes from server.properties because we set different retention bytes to each topic

推荐答案

Zookeeper和Kafka默认将数据存储在/tmp

Zookeeper and Kafka default to store data in /tmp

如果重新启动计算机，/tmp将被清除

If you reboot the machines, /tmp is cleared

否则，如果使用 confluent start 命令，则这不是永久数据.

Otherwise, if you use confluent start command, this is not permanent data.

如果您在未装载任何卷的情况下使用Docker/Kubernetes，这也不是永久的.

If you use Docker/Kubernetes without mounting any volumes, this is also not permanent.

您还可以使用 kafka-topics --describe 命令而不是 zookeeper-shell 命令，因为Zookeeper在即将发布的Kafka版本中将被完全删除

You sould also be using kafka-topics --describe command rather than zookeeper-shell as Zookeeper will be removed completely in upcoming Kafka releases

这篇关于如何在kafka机器重启后仍然保留kafka保留字节和kafka保留段的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！