从JDBC Kafka Connect源中删除事件 [英] Delete events from JDBC Kafka Connect Source

问题描述

我正在研究Kafka Connect JDBC连接器，并专门研究放入该主题的数据的实际格式.

I am playing around with the Kafka Connect JDBC connector and specifically looking at what the actual format of the data that is put onto the topic is.

我已经能够看到数据库的新插入和更新，但是我无法检测到数据库中的删除.

I have been able to see new inserts and updates to the database, but I have not been able to detect deletes from the database.

首先:JDBC源是否支持检测这些更改?我找不到一种或另一种文档.

First: Does the JDBC source support detecting these changes? I can't find documentation one way or another.

如果是，那么实际主题采用什么格式?

If it does, what format does it take on the actual topic?

推荐答案

Confluent JDBC源连接器能够捕获软删除"，其中已删除"行仅被应用程序标记为软删除"，但实际上并未从表中删除.由于行仍然存在，因此连接器可以看到其更改.但是，连接器不能捕获从表中删除的行，因为连接器通过JDBC查询源表，因此无法看到从表中删除的行.

The Confluent JDBC source connector is able to capture "soft deletes", where the "deleted" rows are simply marked as such by your application but are not actually removed from the table. Since the rows are still there, the connector can see their changes. However, the connector is not able to capture rows that are deleted from a table, since the connector queries the source tables via JDBC and thus is unable to see rows that are removed from the tables.

其他连接器能够使用称为"变更数据捕获"或CDC 的技术，通过进入数据库事务日志或预先写入日志来捕获所有变更.每个DBMS不同，因此需要专门为该DBMS编写的连接器.例如， Debezium项目具有适用于MySQL，PostgreSQL和MongoDB的Kafka Connect连接器，并且正在开发适用于Oracle和SQL的连接器.服务器.

Other connectors are able to capture all of the changes by tapping into the database transaction logs or write ahead logs, using techniques known as change data capture, or CDC. Each DBMS is different, and therefore requires a connector written specifically for that DBMS. For example, the Debezium project has Kafka Connect connectors for MySQL, PostgreSQL, and MongoDB, and is working on connectors for Oracle and SQL Server.

这篇关于从JDBC Kafka Connect源中删除事件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！