消费者.如何指定要读取的分区?[卡夫卡] [英] consumer.How to specify partition to read? [kafka]

问题描述

我正在用kafka进行介绍，我想知道当我使用来自主题的消息时如何指定分区.

I am introducing with kafka and I want to know how to specify partition when I consume messages from topic.

我发现了几张这样的照片:

I have found several picture like this:

这意味着1个使用者可以使用多个分区中的消息，但是1个分区可以由单个使用者(在消费者组中)读取

It means that 1 consumer can consume messages from several partitions but 1 partition can be read by single consumer(within consumer group)

我还阅读了一些面向消费者的示例，它看起来像这样:

Also I have read several examples for consumer and it looks like this:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", "consumer-tutorial");
props.put("key.deserializer", StringDeserializer.class.getName());
props.put("value.deserializer", StringDeserializer.class.getName());
KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props); 

和:

consumer.subscribe(Arrays.asList("foo", "bar")); 

2.投票

 try {
      while (running) {
        ConsumerRecords<String, String> records = consumer.poll(1000);
        for (ConsumerRecord<String, String> record : records)
          System.out.println(record.offset() + ": " + record.value());
      }
    } finally {
      consumer.close();
    }

这是如何工作的?我将从哪个分区读取消息?

How does this work? From which partition will I read messages?

推荐答案

有两种方法可以告诉您要使用的主题/分区:

There are two ways to tell what topic/partitions you want to consume: KafkaConsumer#assign() (you specify the partition you want and the offset where you begin) and subscribe (you join a consumer group, and partition/offset will be dynamically assigned by group coordinator depending of consumers in the same consumer group, and may change during runtime)

在两种情况下，您都需要轮询以接收数据.

In both case, you need to poll to receive data.

请参见 https://kafka.apache.org/0110/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html ，尤其是消费者组和主题订阅和手动分区分配

这篇关于消费者.如何指定要读取的分区?[卡夫卡]的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！