向 Kafka Connect 添加连接器 [英] Adding a connector to Kafka Connect

问题描述

我正在使用 Confluent Kafka Docker 镜像，特别是使用这个:https://github.com/confluentinc/cp-docker-images/tree/4.0.x/examples/cp-all-in-one

I am using Confluent Kafka Docker image, specifically using this: https://github.com/confluentinc/cp-docker-images/tree/4.0.x/examples/cp-all-in-one

我想通过以下方式添加 MySQL 连接器:

I want to add the MySQL connector, by:

• 下载连接器 1.5.46 版本 (https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz)
• 使用来自下载的连接器 (mysql-connector-java-5.1.46-bin.jar) 的 jar 安装卷
• 将 CONNECT_PLUGIN_PATH 添加到 docker compose 文件

• downloading the version 1.5.46 of the connector (https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz)
• mounting a volume with a jar that comes out of the downloaded connector (mysql-connector-java-5.1.46-bin.jar)
• adding the CONNECT_PLUGIN_PATH to the docker compose file

基本上，从撰写文件中推断出感兴趣的部分，这是重要的片段:

Basically, extrapolating the part of interest form the compose file, this is the important snippet:

environment:
  <cut>
  CONNECT_PLUGIN_PATH: '/etc/kafka-connect/jars/'
volumes:
  - "./mysql_connector:/etc/kafka-connect/jars/"

JAR 正确存在于 Docker 容器中.

The JAR is correctly present in the Docker container.

但是，当然，如果我尝试使用连接器，我在已启用连接器的列表中找不到.如果我去控制中心，我会看到:

But, of course, if I try to use the connector, I can't find in the list of the enabled connectors. If I go to control center, I see:

[{
    "class": "org.apache.kafka.connect.file.FileStreamSinkConnector",
    "type": "sink",
    "version": "1.0.0-cp1"
}, {
    "class": "org.apache.kafka.connect.file.FileStreamSourceConnector",
    "type": "source",
    "version": "1.0.0-cp1"
}]

推荐答案

您的插件路径需要包含实际 JDBC Connect JAR 文件的路径以及 MySQL JAR

Your plugin path needs to contain the path to the actual JDBC Connect JAR files as well as the MySQL JAR

如果你在这个路径下挂载 MySQL JAR

If you mount the MySQL JAR under this path

/usr/share/java/kafka-connect-jdbc

然后设置CONNECT_PLUGIN_PATH=/usr/share/java，然后您就可以访问所有捆绑的连接器.JDBC 连接器将能够读/写 MySQL 数据.

Then set the CONNECT_PLUGIN_PATH=/usr/share/java, then you have access to all bundled connectors. And the JDBC Connector will be able to read/write MySQL data.

这篇关于向 Kafka Connect 添加连接器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！