JDBC Kafka Connector 能否从多个数据库中拉取数据? [英] Can the JDBC Kafka Connector pull data from multiple databases?

问题描述

我想设置一个 JDBC Kafka 连接器集群，并将它们配置为从运行在同一主机上的多个数据库中提取数据.

I would love to setup a cluster of JDBC Kafka Connectors and configure them to pull from multiple databases running on the same host.

我一直在查看 Kafka Connect 文档，在您配置 JDBC 连接器后，它似乎只能从单个数据库中提取数据.

I've been looking through the Kafka Connect documentation, and it appears that after you configure the JDBC connector that it can only pull data from a single database.

谁能确认一下?

推荐答案

取决于您启动 worker 的模式(独立或分布式):

Depending on the mode you start your workers (standalone or distributed) :

在standalone模式，你可以使用以下命令启动多个jdbc连接器:

In the standalone mode, you can start multiple jdbc connectors by using :

bin/connect-standalone worker.properties connector1.properties [connector2.properties connector3.properties ...]

其中每个 connector.properties 匹配一个数据库.

Where each connector.properties match one database.

更多详情:运行工人

在分布式模式中，首先用:

 bin/connect-distributed worker.properties

然后使用 POST http 请求推送每个配置，例如:

Then push each configuration with POST http requests like :

$ curl -X POST -H "Content-Type: application/json" --data '{"name": "jdbc-source", "config": {"connector.class":"JdbcSourceConnector", "tasks.max":"1", "connection.url":"jdbc:sqlite:test.db", "topic.prefix":"connect-jdbc-test-", "mode":"bulk" }}' http://worker_host:8083/connectors

或者，使用包含 JSON 格式配置的文件

Or, to use a file containing the JSON-formatted configuration

$ curl -X POST -H "Content-Type: application/json" --data @config.json http://worker_host:8083/connectors

这篇关于JDBC Kafka Connector 能否从多个数据库中拉取数据?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！