从 JDBC Kafka Connect 源中删除事件 [英] Delete events from JDBC Kafka Connect Source

问题描述

我正在尝试使用 Kafka Connect JDBC 连接器，并专门查看放入该主题的数据的实际格式是什么.

I am playing around with the Kafka Connect JDBC connector and specifically looking at what the actual format of the data that is put onto the topic is.

我已经能够看到对数据库的新插入和更新，但我无法检测到数据库中的删除.

I have been able to see new inserts and updates to the database, but I have not been able to detect deletes from the database.

第一:JDBC 源是否支持检测这些更改?我无法以某种方式找到文档.

First: Does the JDBC source support detecting these changes? I can't find documentation one way or another.

如果是，它在实际主题上采用什么格式?

If it does, what format does it take on the actual topic?

推荐答案

Confluent JDBC 源连接器 能够捕获软删除"，其中已删除"的行被您的应用程序简单地标记为这样，但实际上并未从表中删除.由于行仍然存在，连接器可以看到它们的变化.但是，连接器无法捕获从表中删除的行，因为连接器通过 JDBC 查询源表，因此无法查看从表中删除的行.

The Confluent JDBC source connector is able to capture "soft deletes", where the "deleted" rows are simply marked as such by your application but are not actually removed from the table. Since the rows are still there, the connector can see their changes. However, the connector is not able to capture rows that are deleted from a table, since the connector queries the source tables via JDBC and thus is unable to see rows that are removed from the tables.

其他连接器能够通过使用称为变更数据捕获或 CDC 的技术，利用数据库事务日志或预写日志来捕获所有变更.每个 DBMS 都不同，因此需要专门为该 DBMS 编写的连接器.例如，Debezium 项目有用于 MySQL、PostgreSQL 和 MongoDB 的 Kafka Connect 连接器，并且正在开发用于 Oracle 和 SQL 的连接器服务器.

Other connectors are able to capture all of the changes by tapping into the database transaction logs or write ahead logs, using techniques known as change data capture, or CDC. Each DBMS is different, and therefore requires a connector written specifically for that DBMS. For example, the Debezium project has Kafka Connect connectors for MySQL, PostgreSQL, and MongoDB, and is working on connectors for Oracle and SQL Server.

这篇关于从 JDBC Kafka Connect 源中删除事件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！