Kafka Connect 不输出 JSON [英] Kafka Connect not outputting JSON
问题描述
我正在使用 JDBC Kafka 连接器将数据从数据库读取到 Kafka.这有效,但它总是以 Avro 格式输出数据,即使我已经指定它应该使用 JSON.我知道这样做是因为当我在 python 中使用来自该主题的消息时,我会在每条消息的顶部看到架构.
I am using the JDBC Kafka Connector to read data from a database into Kafka. That works, but it always outputs data in Avro format even though I've specified that it should use JSON. I know it is doing this because when I consume messages from that topic in python, I see the schema at the top of each message.
我像这样运行连接器:
/usr/bin/connect-standalone /etc/schema-registry/connect-json-standalone.properties /etc/kafka-connect-jdbc/view.properties
connect-json-standalone.properties 文件的内容为:
The content of the connect-json-standalone.properties file is:
bootstrap.servers=localhost:9092
key.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schema.registry.url=http://localhost:8081
key.converter.schemas.enable=true
value.converter=org.apache.kafka.connect.json.JsonConverter
value.converter.schema.registry.url=http://localhost:8081
value.converter.schemas.enable=true
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
# Local storage file for offset data
offset.storage.file.filename=/tmp/connect.offsets
/etc/kafka-connect-jdbc/view.properties 的内容是:
and the content of the /etc/kafka-connect-jdbc/view.properties is:
name=view-small-jdbc-daily
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
tasks.max=1
connection.url=jdbc:teradata://domain.com/charset=UTF8,DBS_PORT=1025,DATABASE=test,USER=***,PASSWORD=***,LOB_SUPPORT=OFF
mode=bulk
table.whitelist=test_table
topic.prefix=view5-
推荐答案
您的配置中有两个问题:
Two problems in your configuration:
- 如果您使用 JSONConverter,则无需配置架构注册表 - 这仅适用于 AvroConverter.所以你可以删除
key.converter.schema.registry.url
和value.converter.schema.registry.url
- 其次,你有
key.converter.schemas.enable=true
和value.converter.schemas.enable=true
,所以你不应该对 JSON包括架构.您指定我们应该包括它.如果您不需要架构,只需将它们设置为false
- If you are using JSONConverter, you don't need to configure a schema registry - this is just for the AvroConverter. So you can remove
key.converter.schema.registry.url
andvalue.converter.schema.registry.url
- Second, you have
key.converter.schemas.enable=true
andvalue.converter.schemas.enable=true
, so you should not be surprised that the JSON includes a schema. You specified that we should include it. If you don't want a schema, just set those tofalse
这篇关于Kafka Connect 不输出 JSON的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!