Kafka如何并行消耗一个主题 [英] Kafka how to consume one topic parallel

问题描述

我阅读了kafka文档，仍然不知道如何并行使用一个主题?

I read kafka document, still don't know how consume one topic parallel?

假设:我有一个主题，例如发生了什么事"(请不要分解此主题)，并且我有很多客户想要使用它.那么我该怎么做，以便多个客户可以并行使用它?我应该使用分区和客户组吗?

Suppose: I have one topic like "something happened" (don't split this topic), and I have many customers that want to consume it. So what should I do, so that multiple customers can consume it parallel? Should I use partitioning and customer groups?

我对此有一个主意，但不确定是否正确.

I have one idea about this, but I'm not sure whether is it right.

针对同一主题创建多个分区，并为一个客户创建一个分区，因此一个生产者必须对这些分区产生相同的分区，并且不同客户组中的每个客户都对吗?

Make many partitions about the same topic, and make one partition to one customer, so one producer must produce the same to these partitions, and every customer in the different customer group, is it right?

推荐答案

使用分区是能够并行使用某个主题的方法.假设您的主题有10个分区，那么同一消费者组中的10个消费者可以读取一个分区.如果您的使用者少于分区，则每个分区将负责多个分区.如果您有更多的消费者使用而不是分区使用，那么将有一些消费者不会分配任何分区，并且除了可以替换另一个已死亡的消费者之外，没有其他事情要做.

Using partitions is the way of being able to parallelize the consumption of a topic. Let´s say you have 10 partitions for your topic, then you can have 10 consumers in the same consumer group reading one partition each. If you have less consumers than partitions, then they would be responsible for more than one partition each. If you have more consumers than partitions, then there would be consumers who would not get any partition assigned to them and have nothing to do except being available to replace another consumer who has died.

这篇关于Kafka如何并行消耗一个主题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！