如何集成 Oracle 和 Kafka [英] How to integrate Oracle and Kafka

问题描述

我一直试图找到在单个 Oracle 11g R2 实例中捕获更改通知的最高效/最有效的方法，并将这些事件传送到 Apache Kafka 队列，但我找不到任何简单的示例或教程沿着这些路线.

I've been trying to find the most efficient/effective way capture change notifications in a single Oracle 11g R2 instance and deliver those events to an Apache Kafka queue, but I haven't been able to find any simple examples or tutorials along these lines.

我在 Oracle 方面看到了一些可能性(Streams，更改数据捕获、触发器(糟糕)等等)，但我仍然不确定哪个是最好的追求.

I've seen some possibilities on the Oracle side (Streams, Change Data Capture,triggers (yuck), etc..), but I'm still not sure which would be best to pursue.

这是一个在 GitHub 上使用 MySQL 和 Kafka 的项目，名为 mypipe，我只是没有看到任何东西与 Oracle 类似.我不确定是否最好专注于为此编写 Oracle 包，或者类似于 mypipe 项目等的层.

Here is a project utilizing MySQL and Kafka on GitHub called mypipe, I just haven't seen anything similar for Oracle. I'm not sure if it would be best to focus writing an Oracle package for this, or a layer similar to the mypipe project, etc. etc..

任何建议、建议或示例将不胜感激.谢谢.

Any recommendations, suggestions or examples would be greatly appreciated. Thank you.

推荐答案

目前只有一种工具是开源的，对数据库的影响很小.这是OpenLogReplicator.

There is currently just one tool which is open source and has minimal impact on the database. This is OpenLogReplicator.

• 许可证是 GPL - 它是完全开源的

• license is GPL - it is fully open source

它对源数据库的影响非常小 - 它不需要许可选项，只需打开源上的补充日志记录(就像所有其他复制工具一样)

it has very low impact on the source database - it requires no licensing options and just turning on supplemental logging on the source (like all other replication tools)

它完全用 C++ 编写 - 因此它具有非常低的延迟和高吞吐量

it is written completely in C++ - so it has very low latency and high throughput

它完全在内存中工作

它支持自 11.2.0.1(11.2、12.1、12.2、18、19)以来的所有 Oracle 数据库版本

it supports all Oracle database versions since 11.2.0.1 (11.2, 12.1, 12.2, 18, 19)

它读取二进制格式的 Oracle Redo 日志并将它们发送到 Kafka.它可以在数据库主机上工作，但您也可以将其配置为使用 sshfs 从另一台主机读取重做日志 - 以最小的数据库负载.

It reads binary format of Oracle Redo logs and sends them to Kafka. It can work on the database host, but you can also configure it to read the redo logs using sshfs from another host - with minimal load of the database.

免责声明 #1:我是此解决方案的作者

disclaimer #1: I am the author of this solution

免责声明 #2:致其他 StackOverflow 用户:请不要删除此答案.这个问题有很多重复.但这是第一个问题，其他重复项应重定向到此处并标记为重复项.不是相反.我已从其他问题中删除了所有其他答案，仅将此答案作为主要答案.

disclaimer #2: to other StackOverflow users: please do not delete this answer. This question has a lot of duplicates. But this is the first question and other duplicates should be redirected here and marked as duplicates. Not the other way. I have deleted all other answers from other questions and just leaving this answer as the primary answer.

这篇关于如何集成 Oracle 和 Kafka的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！