Spring Boot 中多个 Kafka 集群上的多个 KafkaConsumer [英] Multiple KafkaConsumer on multiple Kafka Cluster in Spring Boot

问题描述

想要使用 spring-kafka 在不同集群上从 Spring 启动应用程序创建同构 kafka 消费者.

Want to create homogeneous kafka consumers on different clusters from a Spring boot application using spring-kafka.

即想要为已经定义的类创建一个 Kafka Consumer 对象，该对象侦听动态定义的多个集群.

i.e Want to create a Kafka Consumer object for class defined already which listens to multiple cluster defined dynamically.

例如:让我们说一个包含kafkaconsumer 模板.并且有三个 Kafka 集群集群 1、集群 2、集群 3.应用程序 S 作为聚合器从每个集群产生的数据.这里的解决方案是同一模板的三个 consumers 将分别监听并行中的簇.

e.g: Lets say a Spring boot application S which contains the template for kafkaconsumer. And there are three Kafka Clusters custer1, cluster2, cluster3. The application S act as an aggregator of data produced from each of the cluster. Here the solution would be three consumers of the same template will be listening on individual cluster in parallel.

使用 spring-kafka 是否可以实现上述场景?

Is the above scenario is possible using spring-kafka?

推荐答案

@KafkaListener 不能与多个集群对话.您需要委托给真正的听众...

A @KafkaListener can't talk to multiple clusters. You would need to delegate to the real listener...

@KafkaListener(..., containerFactory="cluster1Factory")
public void listen1(...) {
    this.delegate.listen(...);
}

@KafkaListener(..., containerFactory="cluster2Factory")
public void listen2(...) {
    this.delegate.listen(...);
}

etc.

这篇关于Spring Boot 中多个 Kafka 集群上的多个 KafkaConsumer的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！