MongoDB作为Flink的数据源 [英] MongoDB as datasource to Flink

问题描述

MongoDB 可以用作 Apache Flink 的数据源吗?

使用 No-SQL数据库作为数据源的Apache Flink的本机实现是什么?

What is the native implementation of Apache Flink to use No-SQL Database as data source?

推荐答案

当前，Flink没有专用的连接器可从MongoDB中读取.您可以执行以下操作:

Currently, Flink does not have a dedicated connector to read from MongoDB. What you can do is the following:

• 使用StreamExecutionEnvironment.createInput并使用Flink的包装器输入格式为MongoDB提供Hadoop输入格式
• 通过实现SourceFunction/ParallelSourceFunction
来实现自己的MongoDB源

• Use StreamExecutionEnvironment.createInput and provide a Hadoop input format for MongoDB using Flink's wrapper input format
• Implement your own MongoDB source via implementing SourceFunction/ParallelSourceFunction

前者应该给您至少一次处理保证，因为一旦恢复，MongoDB集合将被完全重新读取.根据MongoDB客户端的功能，您可能可以使用后一种方法实施一次精确的处理保证.

The former should give you at-least-once processing guarantees since the MongoDB collection is completely re-read in case of a recovery. Depending on the functionality of the MongoDB client, you might be able to implement exactly-once processing guarantees with the latter approach.

这篇关于MongoDB作为Flink的数据源的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！