Kafka Connect不输出JSON [英] Kafka Connect not outputting JSON

问题描述

我正在使用JDBC Kafka Connector将数据库中的数据读取到Kafka中.那行得通，但是即使我已指定应使用JSON，它也始终以Avro格式输出数据.我知道这样做是因为在python中使用来自该主题的消息时，我会在每条消息的顶部看到架构.

I am using the JDBC Kafka Connector to read data from a database into Kafka. That works, but it always outputs data in Avro format even though I've specified that it should use JSON. I know it is doing this because when I consume messages from that topic in python, I see the schema at the top of each message.

我这样运行连接器:

/usr/bin/connect-standalone /etc/schema-registry/connect-json-standalone.properties /etc/kafka-connect-jdbc/view.properties

connect-json-standalone.properties文件的内容为:

The content of the connect-json-standalone.properties file is:

bootstrap.servers=localhost:9092

key.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schema.registry.url=http://localhost:8081
key.converter.schemas.enable=true
value.converter=org.apache.kafka.connect.json.JsonConverter
value.converter.schema.registry.url=http://localhost:8081
value.converter.schemas.enable=true

internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false

# Local storage file for offset data
offset.storage.file.filename=/tmp/connect.offsets

/etc/kafka-connect-jdbc/view.properties的内容为:

and the content of the /etc/kafka-connect-jdbc/view.properties is:

name=view-small-jdbc-daily
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
tasks.max=1
connection.url=jdbc:teradata://domain.com/charset=UTF8,DBS_PORT=1025,DATABASE=test,USER=***,PASSWORD=***,LOB_SUPPORT=OFF
mode=bulk
table.whitelist=test_table
topic.prefix=view5-

推荐答案

您的配置中存在两个问题:

Two problems in your configuration:

1. 如果使用的是JSONConverter，则无需配置架构注册表-这仅适用于AvroConverter.因此，您可以删除key.converter.schema.registry.url和value.converter.schema.registry.url
2. 第二，您拥有key.converter.schemas.enable=true和value.converter.schemas.enable=true，因此JSON包含架构就不会感到惊讶.您指定了我们应该包括它.如果您不想要模式，只需将其设置为false

1. If you are using JSONConverter, you don't need to configure a schema registry - this is just for the AvroConverter. So you can remove key.converter.schema.registry.url and value.converter.schema.registry.url
2. Second, you have key.converter.schemas.enable=true and value.converter.schemas.enable=true, so you should not be surprised that the JSON includes a schema. You specified that we should include it. If you don't want a schema, just set those to false

这篇关于Kafka Connect不输出JSON的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！