重置JDBC Kafka Connector以便从时间的开头开始拉行吗? [英] Reset the JDBC Kafka Connector to start pulling rows from the beginning of time?

问题描述

Kafka连接器可以利用主键和时间戳来确定需要处理的行.

The Kafka Connector can make use of a primary key and a timestamp to determine which rows need to be processed.

我正在寻找一种重置连接器的方法，以便从时间开始进行处理.

I'm looking for a way to reset the Connector so that it will process from the beginning of time.

推荐答案

因为要求是在分布式模式下运行，所以最简单的方法是将连接器名称更新为新值.这将提示您在connect-offsets主题中输入一个新条目，因为它看起来像是一个全新的连接器.然后，连接器应重新开始读取，就好像尚未将任何内容写入Kafka一样.您还可以手动将逻辑删除消息发送到与该特定连接器相关联的connect-offsets主题中的键，但是重命名比处理它要容易得多.此方法适用于所有源连接器，不仅适用于此处描述的JDBC.

Because the requirement is to run in distributed mode, the easiest thing to do is to update the connector name to a new value. This will prompt a new entry to be made into the connect-offsets topic as it looks like a totally new connector. Then the connector should start reading again as if nothing has been written to Kafka yet. You could also manually send a tombstone message to the key in the connect-offsets topic associated with that particular connector, but renaming is much easier than dealing with that. This method applies to all source connectors, not only the JDBC one described here.

这篇关于重置JDBC Kafka Connector以便从时间的开头开始拉行吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！