如何整合Oracle和Kafka [英] How to integrate Oracle and Kafka

问题描述

我一直试图找到一种最有效的方式来捕获单个Oracle 11g R2实例中的更改通知，并将这些事件传递到Apache Kafka队列中，但是我却找不到任何简单的示例或教程.遵循这些原则.

I've been trying to find the most efficient/effective way capture change notifications in a single Oracle 11g R2 instance and deliver those events to an Apache Kafka queue, but I haven't been able to find any simple examples or tutorials along these lines.

我已经在Oracle方面看到了一些可能性(流，

I've seen some possibilities on the Oracle side (Streams, Change Data Capture,triggers (yuck), etc..), but I'm still not sure which would be best to pursue.

这是一个在GitHub上使用MySQL和Kafka的项目，名为 mypipe ，我只是什么都没看到与Oracle类似.我不确定是否最好集中精力为此编写一个Oracle软件包，或者是类似于mypipe项目的一层等等.

Here is a project utilizing MySQL and Kafka on GitHub called mypipe, I just haven't seen anything similar for Oracle. I'm not sure if it would be best to focus writing an Oracle package for this, or a layer similar to the mypipe project, etc. etc..

任何建议，建议或示例将不胜感激.谢谢.

Any recommendations, suggestions or examples would be greatly appreciated. Thank you.

推荐答案

当前只有一种工具是开源的，对数据库的影响最小.这是 OpenLogReplicator .

There is currently just one tool which is open source and has minimal impact on the database. This is OpenLogReplicator.

• 许可是GPL-它是完全开源的

• license is GPL - it is fully open source

它对源数据库的影响很小-它不需要许可选项，只需打开源上的补充日志记录(就像所有其他复制工具一样)

it has very low impact on the source database - it requires no licensing options and just turning on supplemental logging on the source (like all other replication tools)

它是完全用C ++编写的-因此它具有非常低的延迟和高吞吐量

it is written completely in C++ - so it has very low latency and high throughput

它完全可以在内存中工作

it works completely in memory

它支持自11.2.0.1(11.2、12.1、12.2、18、19)以来的所有Oracle数据库版本

it supports all Oracle database versions since 11.2.0.1 (11.2, 12.1, 12.2, 18, 19)

它读取Oracle Redo日志的二进制格式并将其发送到Kafka.它可以在数据库主机上运行，​​但是您也可以将其配置为使用sshfs从另一台主机读取重做日志，而数据库的负载却最小.

It reads binary format of Oracle Redo logs and sends them to Kafka. It can work on the database host, but you can also configure it to read the redo logs using sshfs from another host - with minimal load of the database.

免责声明#1:我是此解决方案的作者

disclaimer #1: I am the author of this solution

免责声明#2:对其他StackOverflow用户:请不要删除此答案.这个问题有很多重复之处.但这是第一个问题，其他重复项应在此处重定向并标记为重复项.别无选择.我已从其他问题中删除了所有其他答案，而仅将此答案作为主要答案.

disclaimer #2: to other StackOverflow users: please do not delete this answer. This question has a lot of duplicates. But this is the first question and other duplicates should be redirected here and marked as duplicates. Not the other way. I have deleted all other answers from other questions and just leaving this answer as the primary answer.

这篇关于如何整合Oracle和Kafka的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！