测试“故障转移"关于卡夫卡 [英] Testing "fail-over" on Kafka
问题描述
操作系统:Windows 10
OS: Windows 10
3 ZooKeeper 实例(使用 v3.5.6 和 v.3.4.14 测试):
(1) apache-zookeeper-3.5.6-bin_1
(2) apache-zookeeper-3.5.6-bin_2 (1 的副本)
(3) apache-zookeeper-3.5.6-bin_3 (1 的副本)
3 ZooKeeper instances downloaded from Apache(tested with v3.5.6 and v.3.4.14):
(1) apache-zookeeper-3.5.6-bin_1
(2) apache-zookeeper-3.5.6-bin_2 (Copy of 1)
(3) apache-zookeeper-3.5.6-bin_3 (Copy of 1)
zoo.cfg:
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zookeeper_3.4.14_1
clientPort=2181
admin.serverPort=10081
server.1=localhost:2881:3881
server.2=localhost:2882:3882
server.3=localhost:2883:3883
4lw.commands.whitelist=*
zoo.cfg:
...
dataDir=/tmp/zookeeper_3.4.14_2
clientPort=2182
admin.serverPort=10082
...
zoo.cfg:
...
dataDir=/tmp/zookeeper_3.4.14_3
clientPort=2183
admin.serverPort=10083
...
dataDir 中的 myid 文件，值分别为 1,2 和 3
2 Kafka 实例:
(1) kafka_2.12-2.3.0_1
(2) kafka_2.12-2.3.0_2 (1 的副本)
2 Kafka instances:
(1) kafka_2.12-2.3.0_1
(2) kafka_2.12-2.3.0_2 (Copy of 1)
server.properties:
...
broker.id=1
listeners=PLAINTEXT://:9091
log.dirs=/tmp/kafka-logs-1
zookeeper.connect=localhost:2181,localhost:2182,localhost:2183
...
server.properties:
...
broker.id=2
listeners=PLAINTEXT://:9092
log.dirs=/tmp/kafka-logs-2
zookeeper.connect=localhost:2181,localhost:2182,localhost:2183
...
春天
spring-boot-starter-* 2.2.0.RELEASE
spring-kafka-2.3.1.RELEASE
Spring
spring-boot-starter-* 2.2.0.RELEASE
spring-kafka-2.3.1.RELEASE
======================================================================
=====================================================================
与设置 1 相同，唯一的区别是我使用的是 ZooKeeper，而不是使用从 Apache 下载的 ZooKeeperKafka 自带的.
Same as set-up 1, the only difference being that instead of using the ZooKeeper downloaded from Apache, i am using the ZooKeeper that comes with Kafka.
======================================================================
=====================================================================
问题是当我放下 1 个 Kafka 时:
The issue is that when i bring 1 Kafka down:
=> 设置 1 将 not 故障转移，这意味着当我生成一条消息时，正在运行的 Kafka 没有接收到消息=> 设置 2 将进行故障转移，这意味着当我生成一条消息时，启动的 Kafka 将收到消息
=> Set-up 1 will not fail-over, meaning that when i produce a message, the Kafka that is up is not receiving the message
=> Set-up 2 will fail-over, meaning that when i produce a message, the Kafka that is up will receive the message
你们觉得设置 1 有什么问题吗?
Do you guys see anything wrong with Set-up 1?
P.S 如果您需要更多详细信息，我很乐意提供.
P.S If you need more details, i am happy to provide.
推荐答案
如果对某人有帮助:
我有 2 个 Kafka 实例，但是我创建的任何主题的 replication-factor 都是 1(由于我对其含义的误解/误解).
I had 2 Kafka instances, but my replication-factor for any topic created was 1(due to my misunderstanding/misinterpretation of its meaning).
这意味着，在创建主题时，主题将在 Kafka-1 或 Kafka-2 中创建，而不是同时在两者中创建.因此，当我尝试进行故障转移时，故障转移将失败，具体取决于我正在写入的主题以及被关闭的 Kafka.
What this means is that, at the time of topic creation, the topic/s will be either created in Kafka-1 or Kafka-2, not both. As such, when i tried to do a fail-over, the fail-over will fail, depending on which topic i am writing to, and which Kafka was brought down.
简而言之，如果您有 X 个 Kafka 实例，则主题 replication-factor 需要为 X(offsets.topic.replication.factor)
In short, if you have X Kafka instances, topic replication-factor needs to be X (Same goes for offsets.topic.replication.factor)
这篇关于测试“故障转移"关于卡夫卡的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
