将主题列表从应用程序 yml 传递到 KafkaListener [英] Pass list of topics from application yml to KafkaListener

问题描述

我有以下 application.yml:

I have the following application.yml:

service:
  kafka:
    groupId: 345
    consumer: 
      topics: 
        -
          name: response    
    producer: 
      topics: 
        -
          name: request1 
          num-partitions: 5
          replication-factor: 1 
        -
          name: request2 
          num-partitions: 3
          replication-factor: 1  

如何使用 spel 访问主题名称列表以传递给 KafkaListener 注释?

How can I access the list of topic names using spel for passing to KafkaListener annotation?

@KafkaListener(topics = "#{'${service.kafka.consumer.topics.name}'}", containerFactory = "kafkaListenerContainerFactory")
public void receive(String payload, @Header(KafkaHeaders.RECEIVED_TOPIC)String topic) {

推荐答案

使用配置属性和集合投影...

Use configuration properties and collection projection...

@ConfigurationProperties("service.kafka.producer")
@Component
public class ConfigProps {

    List<Topic> topics = new ArrayList<>();

    public List<Topic> getTopics() {
        return this.topics;
    }

    public void setTopics(List<Topic> topics) {
        this.topics = topics;
    }

    @Override
    public String toString() {
        return "ConfigProps [topics=" + this.topics + "]";
    }

    public static class Topic {

        private String name;

        private int numPartitions;

        private short replicationFactor;

        public String getName() {
            return this.name;
        }

        public void setName(String name) {
            this.name = name;
        }

        public int getNumPartitions() {
            return this.numPartitions;
        }

        public void setNumPartitions(int numPartitions) {
            this.numPartitions = numPartitions;
        }

        public short getReplicationFactor() {
            return this.replicationFactor;
        }

        public void setReplicationFactor(short replicationFactor) {
            this.replicationFactor = replicationFactor;
        }

        @Override
        public String toString() {
            return "Topic [name=" + this.name + ", numPartitions=" + this.numPartitions + ", replicationFactor="
                    + this.replicationFactor + "]";
        }

    }

}

和

@SpringBootApplication
public class So52741016Application {

    public static void main(String[] args) {
        SpringApplication.run(So52741016Application.class, args);
    }

    @KafkaListener(groupId = "${service.kafka.groupId}", topics = "#{configProps.topics.![name]}")
    public void listener(String in) {

    }

    @Bean
    public SmartLifecycle createTopics(KafkaAdmin admin, ConfigProps props) {
        return new SmartLifecycle() {

            @Override
            public int getPhase() {
                return Integer.MIN_VALUE;
            }

            @Override
            public void stop() {
            }

            @Override
            public void start() {
                try (AdminClient client = AdminClient.create(admin.getConfig())) {
                    CreateTopicsResult createTopics = client.createTopics(props.topics.stream()
                        .map(t -> new NewTopic(t.getName(), t.getNumPartitions(), t.getReplicationFactor()))
                        .collect(Collectors.toList()));
                    createTopics.all().get();
                }
                catch (Exception e) {
//                  e.printStackTrace();
                }
            }

            @Override
            public boolean isRunning() {
                return false;
            }

            @Override
            public void stop(Runnable callback) {
            }

            @Override
            public boolean isAutoStartup() {
                return true;
            }
        };
    }

}

和

2018-10-10 11:20:25.813 INFO 14979 --- [ntainer#0-0-C-1] osklKafkaMessageListenerContainer :分配的分区:[request1-4, request2-0, request1-0, request2-1、请求1-1、请求2-2、请求1-2、请求1-3]

2018-10-10 11:20:25.813 INFO 14979 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer : partitions assigned: [request1-4, request2-0, request1-0, request2-1, request1-1, request2-2, request1-2, request1-3]

当然，这只是制作人的话题，但你可以这样处理.

Of course, this is only the producer topics, but you can handle them all this way.

这篇关于将主题列表从应用程序 yml 传递到 KafkaListener的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！