如何查看kafka头文件 [英] how to view kafka headers

问题描述

我们正在使用标题向 Kafka 发送消息org.apache.kafka.clients.producer.ProducerRecord

We are sending message with headers to Kafka using org.apache.kafka.clients.producer.ProducerRecord

public ProducerRecord(String topic, Integer partition, K key, V value, Iterable<Header> headers) {
    this(topic, partition, (Long)null, key, value, headers);
}

如何使用命令实际查看这些标题.kafka-console-consumer.sh 只显示有效负载，没有标题.

How can I actually see these headers using command. kafka-console-consumer.sh only shows me payload and no headers.

推荐答案

您可以使用优秀的 kafkacat 工具.

You can use the excellent kafkacat tool.

示例命令:

kafkacat -b kafka-broker:9092 -t my_topic_name -C \
  -f '\nKey (%K bytes): %k
  Value (%S bytes): %s
  Timestamp: %T
  Partition: %p
  Offset: %o
  Headers: %h\n'

示例输出:

Key (-1 bytes):
  Value (13 bytes): {foo:"bar 5"}
  Timestamp: 1548350164096
  Partition: 0
  Offset: 34
  Headers: __connect.errors.topic=test_topic_json,__connect.errors.partition=0,__connect.errors.offset=94,__connect.errors.connector.name=file_sink_03,__connect.errors.task.id=0,__connect.errors.stage=VALU
E_CONVERTER,__connect.errors.class.name=org.apache.kafka.connect.json.JsonConverter,__connect.errors.exception.class.name=org.apache.kafka.connect.errors.DataException,__connect.errors.exception.message=Co
nverting byte[] to Kafka Connect data failed due to serialization error: ,__connect.errors.exception.stacktrace=org.apache.kafka.connect.errors.DataException: Converting byte[] to Kafka Connect data failed
 due to serialization error:

kafkacat 头选项仅在 kafkacat 的最新版本中可用；如果您当前的版本不包含它，您可能希望自己从 master 分支构建.

The kafkacat header option is only available in recent builds of kafkacat; you may want to build from master branch yourself if your current version doesn't include it.

您也可以从 Docker 运行 kafkacat:

You can also run kafkacat from Docker:

docker run --rm edenhill/kafkacat:1.5.0 \
      -b kafka-broker:9092 \
      -t my_topic_name -C \
      -f '\nKey (%K bytes): %k
  Value (%S bytes): %s
  Timestamp: %T
  Partition: %p
  Offset: %o
  Headers: %h\n'

如果您使用 Docker，请记住如何访问 Kafka 代理的网络影响.

If you use Docker bear in mind the network implications of how to reach the Kafka broker.

这篇关于如何查看kafka头文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！