Kafka使用者无法使用引导服务器名称来使用消息 [英] Kafka consumer not able to consume messages using bootstrap server name

问题描述

使用引导服务器(即Kafka服务器)消费消息时，我遇到了问题.知道为什么没有zookeeper不能消费消息吗?

I am facing an issue while consuming message using the bootstrap-server i.e. Kafka server. Any idea why is it not able to consume messages without zookeeper?

• Kafka版本:kafka_2.11-1.0.0
• Zookeeper版本:kafka_2.11-1.0.0
• Zookeeper主机和端口:zkp02.mp.com:2181
• Kafka主机和端口:kfk03.mp.com:9092

产生一些消息:

[kfk03.mp.com ~]$ /bnsf/kafka/bin/kafka-console-producer.sh --broker-list kfk03.mp.com:9092 --topic test
>hi
>hi

如果我提供-bootstrap-server ，消费者将无法使用消息:

Consumer not able to consume messages if I give –-bootstrap-server:

[kfk03.mp.com ~]$
/bnsf/kafka/bin/kafka-console-consumer.sh --bootstrap-server kfk03.mp.com:9092 --topic test --from-beginning

在提供-zookeeper 服务器而不是-bootstrap-server -:

Consumer able to consume messages when --zookeeper server is given instead of --bootstrap-server -:

[kfk03.mp.com ~]$ /bnsf/kafka/bin/kafka-console-consumer.sh --zookeeper zkp02.mp.com:2181 --topic test --from-beginning

Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].

{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
hi
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
{"properties": {"messageType": "test", "sentDateTime": "2018-02-25T21:46:00.000+0000"}, "name": "Uttam Anand", "age": 29}
hi
hi
uttam
hi
hi
hi
hello
hi
^CProcessed a total of 17 messages

推荐答案

虽然使用bootstrap-server参数消耗来自kafka的消息，但连接是通过kafka服务器而不是zookeeper进行的.Kafka经纪人将抵销详细信息存储在__consumer_offsets主题中.

While consuming messages from kafka using bootstrap-server parameter, the connection happens via the kafka server instead of zookeeper. Kafka broker stores offset details in __consumer_offsets topic.

检查主题列表中是否存在__consumer_offsets.如果不存在，请检查kafka日志以查找原因.

Check if __consumer_offsets is present in your topics list. If it's not there, check kafka logs to find the reason.

我们遇到了类似的问题.在我们的情况下，由于以下错误，未创建__consumer_offsets:

We faced a similar issue. In our case the __consumer_offsets was not created because of the following error:

ERROR [KafkaApi-1001] Number of alive brokers '1' does not meet the required replication factor '3' for the offsets topic (configured via 'offsets.topic.replication.factor').

这篇关于Kafka使用者无法使用引导服务器名称来使用消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！