即使在 kafka 机器重新启动后,如何保留 kafka 保留字节和 kafka 保留段 [英] how to remaine the kafka retentions-bytes and kafka retention-segment even after kafka machine reboot
问题描述
我们为主题设置retention bytes
值 - 104857600
- topic_test
we set retention bytes
value - 104857600
for topic - topic_test
[root@confluent01 ~]# kafka-topics --zookeeper localhost:2181 --alter --topic topic_test --config retention.bytes=104857600
WARNING: Altering topic configuration from this script has been deprecated and may be removed in future releases.
Going forward, please use kafka-configs.sh for this functionality
Updated config for topic "topic_test".
现在我们验证来自zookeeper的保留字节
:
Now we verify the retention bytes
from the zookeeper:
[root@confluent01 ~]# zookeeper-shell confluent01:2181 get /config/topics/topic_test
Connecting to confluent1:2181
{"version":1,"config":{"retention.bytes":"104857600"}}
cZxid = 0xb30a00000038
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
ctime = Mon Jun 29 11:42:30 GMT 2020
mZxid = 0xb31100008978
mtime = Wed Jul 22 19:22:20 GMT 2020
pZxid = 0xb30a00000038
cversion = 0
dataVersion = 7
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 54
numChildren = 0
现在我们对 kafka confluent01 机器执行了reboot
now we performed reboot
to the kafka confluent01 machine
机器启动并成功启动kafka服务后,我们再次检查zookeeper的retention-bytes
:
after machines started and kafka service start successfully , we checked again the retention-bytes
from zookeeper:
但是现在(机器重启后)我们可以看到retention bytes
没有在zookeeper中配置
but now ( after machine reboot ) we can see that the retention bytes
isn't configured in zookeeper
[root@confluent01 ~]#zookeeper-shell confluent01:2181 get /config/topics/topic_test
Connecting to confluent1:2181
WATCHER::
WatchedEvent state:SyncConnected type:None path:null no retention bytes value
{"version":1,"config":{}}
cZxid = 0xb30a00000038
ctime = Mon Jun 29 11:42:30 GMT 2020
mZxid = 0xb3110000779b
mtime = Wed Jul 22 14:09:19 GMT 2020
pZxid = 0xb30a00000038
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 25
numChildren = 0
问题是 - 如何在 kafka 机器重启后保留 retention bytes
?
the question is - how to remain the retention bytes
even after restart of kafka machine ?
注意 - 我们不想使用 server.properties
中的保留字节因为我们为每个主题设置了不同的保留字节
NOTE - we not want to use the retention bytes from server.properties
because we set different retention bytes to each topic
推荐答案
Zookeeper 和 Kafka 默认将数据存储在/tmp
Zookeeper and Kafka default to store data in /tmp
如果您重新启动机器,/tmp 将被清除
If you reboot the machines, /tmp is cleared
否则,如果使用confluent start
命令,这不是永久数据.
Otherwise, if you use confluent start
command, this is not permanent data.
如果您在没有安装任何卷的情况下使用 Docker/Kubernetes,这也不是永久性的.
If you use Docker/Kubernetes without mounting any volumes, this is also not permanent.
您也应该使用 kafka-topics --describe
命令而不是 zookeeper-shell
,因为 Zookeeper 将在即将发布的 Kafka 版本中完全删除
You sould also be using kafka-topics --describe
command rather than zookeeper-shell
as Zookeeper will be removed completely in upcoming Kafka releases
这篇关于即使在 kafka 机器重新启动后,如何保留 kafka 保留字节和 kafka 保留段的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!