检索 ID 为 1 的 Avro 架构时出错，未找到主题.；错误代码:40401 [英] Error retrieving Avro schema for id 1, Subject not found.; error code: 40401
问题描述
Caused by: org.apache.kafka.common.errors.SerializationException: Error retrieving Avro schema for id 1
Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Subject not found.; error code: 40401
Confluent 4.1.0 版
Confluent Version 4.1.0
我使用 KTable 从几个主题(topic_1、topic_2)中获取数据，连接数据，然后使用 KStream 将数据推送到另一个主题(topic_out).(Ktable.toStream())
I am consuming data from a couple of topics(topic_1, topic_2) using KTable, joining the data and then pushing the data onto another topic(topic_out) using KStream. (Ktable.toStream())
数据为avro格式
当我使用检查架构时
curl -X GET http://localhost:8081/subjects/
我找到了
topic_1-value
topic_1-key
topic_2-value
topic_2-key
topic_out-value
但是没有带有 topic_out-key 的主题.为什么没有创建?
but there is no subject with topic_out-key. Why is it not created?
topic_out 的输出:
output from topic_out:
kafka-avro-console-consumer --bootstrap-server localhost:9092 --from-beginning --property print.key=true --topic topic_out
"code1 " {"code":{"string":"code1 "},"personid":{"string":"=NA="},"agentoffice":{"string":"lic1 "},"status":{"string":"a"},"sourcesystem":{"string":"ILS"},"lastupdate":{"long":1527240990138}}
我可以看到正在生成的密钥，但没有密钥的主题.
I can see the key being generated, but no subject for key.
为什么需要带键的主题?
我正在将此主题提供给另一个连接器 (hdfs-sink) 以将数据推送到 hdfs，但它失败并显示以下错误
Why is subject with key required?
I am feeding this topic to another connector (hdfs-sink) to push the data to hdfs but it fails with below error
Caused by: org.apache.kafka.common.errors.SerializationException: Error retrieving Avro schema for id 5\nCaused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Subject not found.; error code: 40401
当我查看 schema-registry.logs 时，我可以看到:
when I look at the schema-registry.logs, I can see:
[2018-05-24 15:40:06,230] INFO 127.0.0.1 - -
[24/May/2018:15:40:06 +0530] "POST /subjects/topic_out-key?deleted=true HTTP/1.1" 404 51 9 (io.confluent.rest-utils.requests:77)
知道为什么没有创建主题 topic_out-key 吗?
any idea why the subject topic_out-key not being created?
推荐答案
知道为什么没有创建主题 topic_out-key
any idea why the subject topic_out-key not being created
因为您的 Kafka Streams 输出的 Key 是一个字符串，而不是一个 Avro 编码的字符串.
Because the Key of your Kafka Streams output is a String, not an Avro encoded string.
您可以验证使用 kafka-console-consumer 代替并添加 --property print.value=false 并且与相同的命令相比没有看到任何特殊字符您确实打印了值(这表明数据是二进制 Avro)
You can verify that using kafka-console-consumer instead and adding --property print.value=false and not seeing any special characters compared to the same command when you do print the value (this is showing the data is binary Avro)
从 Kafka Connect 开始，您必须将 Kafka 的 StringConverter 类用于 key.converter 属性，而不是 Confluent Avro 类
From Kafka Connect, you must use Kafka's StringConverter class for key.converter property rather than the Confluent Avro one
这篇关于检索 ID 为 1 的 Avro 架构时出错，未找到主题.；错误代码:40401的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
