num.stream.threads创建空闲线程 [英] num.stream.threads creating idle threads

问题描述

我有一个带有两个主题的spring boot kafka流应用程序，考虑主题A和B.主题A有16个分区，主题B有1个分区.考虑将应用程序部署在1个实例中，该实例的num.stream.threads = 16. 我运行kafka-consumer-groups.bat命令来检查如何将线程分配给组中的分区，得到以下输出.主题A和B分配了16个线程，其中主题B中的14个线程处于空闲状态.

I have a spring boot kafka stream application with 2 topics consider topics A and B.Topic A has 16 partition and Topic B has 1 partition.Consider the application is deployed in 1 instance having num.stream.threads=16. I ran kafka-consumer-groups.bat command to check how the threads are assigned to the partition in group,got the following output.Topic A and B assigned with 16 threads where 14 threads in topic B is idle.

kafka-consumer-groups.bat --bootstrap-server 0.0.0.0:9092 --group <topic-AGroupName>  -describe
TOPIC     PARTITION      CLIENT-ID
topic-A    0          topic-AGroupStreamThread-429-consumer
topic-A    10         topic-AGroupStreamThread-422-consumer
topic-A    6          topic-AGroupStreamThread-425-consumer
topic-A    14         topic-AGroupStreamThread-431-consumer
topic-A    7          topic-AGroupStreamThread-420-consumer
topic-A    8          topic-AGroupStreamThread-428-consumer
topic-A    11         topic-AGroupStreamThread-430-consumer
topic-A    4          topic-AGroupStreamThread-417-consumer
topic-A    2          topic-AGroupStreamThread-427-consumer
topic-A    1          topic-AGroupStreamThread-418-consumer
topic-A    15         topic-AGroupStreamThread-421-consumer
topic-A    3          topic-AGroupStreamThread-424-consumer
topic-A    9          topic-AGroupStreamThread-419-consumer
topic-A    12         topic-AGroupStreamThread-426-consumer
topic-A    13         topic-AGroupStreamThread-432-consumer
topic-A    5          topic-AGroupStreamThread-423-consumer

kafka-consumer-groups.bat --bootstrap-server 0.0.0.0:9092 --group <topic-BGroupName>  -describe
TOPIC                               PARTITION                    CLIENT-ID
topic-BGroup-result-repartition     0            topic-BGroupStreamThread-683-consumer
topic-B                             0            topic-BGroupStreamThread-678-consumer
-                                   -            topic-BGroupStreamThread-685-consumer
-                                   -            topic-BGroupStreamThread-675-consumer
-                                   -            topic-BGroupStreamThread-673-consumer
-                                   -            topic-BGroupStreamThread-680-consumer
-                                   -            topic-BGroupStreamThread-688-consumer
-                                   -            topic-BGroupStreamThread-674-consumer
-                                   -            topic-BGroupStreamThread-679-consumer
-                                   -            topic-BGroupStreamThread-677-consumer
-                                   -            topic-BGroupStreamThread-687-consumer
-                                   -            topic-BGroupStreamThread-686-consumer
-                                   -            topic-BGroupStreamThread-676-consumer
-                                   -            topic-BGroupStreamThread-684-consumer
-                                   -            topic-BGroupStreamThread-682-consumer
-                                   -            topic-BGroupStreamThread-681-consumer

如何避免主题B中的空闲线程，或者是否有任何选项可用于为每个主题设置num.stream.threads?

How to avoid the idle threads in topic-B or is there any options available to set num.stream.threads per topic?

推荐答案

您可以在实际的输入绑定上设置并发属性.例如spring.cloud.stream.kafka.streams.bindings.<input-binding-name>.consumer.concurrency.每个输入绑定将其映射到幕后的num.stream.threads.看看是否可行，如果您发现任何问题，请告诉我们.

You can set the concurrency property on the actual input binding. For e.g. spring.cloud.stream.kafka.streams.bindings.<input-binding-name>.consumer.concurrency. This gets mapped to num.stream.threads behind the scenes per input binding. See if that works and please let us know if you see any issues.

这篇关于num.stream.threads创建空闲线程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！