是否可以使用 MirrorMaker2 复制没有别名前缀的 kafka 主题 [英] Is it possible to replicate kafka topics without alias prefix with MirrorMaker2
问题描述
我正在尝试在 2 个集群之间设置复制,但不想更改主题名称.例如,如果我有一个名为some_topic"的主题,它会自动复制到cluster1.some_topic",我很确定这可以完成,但还没有找到正确的配置来改变它
I'm trying to set up replication between 2 clusters but don't want want the topic names to be changed. for example if i have a topic called "some_topic" it is automatically replicated to "cluster1.some_topic", I'm pretty sure this can be done but haven't found the correct config to change this
我当前的配置mirrormaker2.properties"
My current config "mirrormaker2.properties"
# Sample MirrorMaker 2.0 top-level configuration file
# Run with ./bin/connect-mirror-maker.sh connect-mirror-maker.properties
# specify any number of cluster aliases
clusters = cluster1, cluster2
# connection information for each cluster
cluster1.bootstrap.servers = host1:9092,host2:9092,host3:9092
cluster2.bootstrap.servers = rep_host1:9092,rep_host2:9092,rep_host3:9092
# enable and configure individual replication flows
cluster1->cluster2.enabled = true
cluster1->cluster2.topics = sometopic.*
# customize as needed
# replication.policy.separator = _
# sync.topic.acls.enabled = false
# emit.heartbeats.interval.seconds = 5
供参考:
- https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP-382:MirrorMaker2.0-RunningastandaloneMirrorMakerconnector
- https://kafka.apache.org/24/javadoc/index.html?constant-values.html
推荐答案
我认为上面的答案不合适.
I think the answer above is inappropriate.
在 Mirror Maker 2.0 中,如果要保持主题不变,则必须实现 ReplicationPolicy.
In Mirror Maker 2.0, if you want to keep the topic unmodified, you have to implement ReplicationPolicy.
您可以参考DefaultReplicationPolicy.class,然后覆盖formatRemoteTopic()
,然后删除sourceClusterAlias + separator
.最后在mm2.properties
You can refer to DefaultReplicationPolicy.class, and then override formatRemoteTopic()
, after that you have to remove sourceClusterAlias + separator
. In the end, configure replication.policy.class
in the mm2.properties
我定义了MigrationReplicationPolicy.class
replication.policy.class = org.apache.kafka.connect.mirror.MigrationReplicationPolicy
你应该看到MirrorClientConfig,class
,我知道你会明白
You should see MirrorClientConfig,class
, I know that you will understand
这篇关于是否可以使用 MirrorMaker2 复制没有别名前缀的 kafka 主题的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!