Apache Beam 和 Apache Nifi 的区别 [英] Difference between Apache Beam and Apache Nifi

问题描述

Apache Beam 和 Apache Nifi 的用例是什么?似乎它们都是数据流引擎.如果两者都有相似的用例，两者哪个更好?

What are the use cases for Apache Beam and Apache Nifi? It seems both of them are data flow engines. In case both have similar use case, which of the two is better?

推荐答案

Apache Beam 是 Apache Flink、Apache Spark(流)、Apache Apex 和 Apache Storm 等流处理系统的抽象层.它允许您针对标准 API 编写代码，然后使用任何底层平台执行代码.因此，理论上，如果您针对 Beam API 编写代码，则该代码无需任何代码更改即可在 Flink 或 Spark Streaming 上运行.

Apache Beam is an abstraction layer for stream processing systems like Apache Flink, Apache Spark (streaming), Apache Apex, and Apache Storm. It lets you write your code against a standard API, and then execute the code using any of the underlying platforms. So theoretically, if you wrote your code against the Beam API, that code could run on Flink or Spark Streaming without any code changes.

Apache NiFi 是一种数据流工具，专注于在系统之间移动数据，从使用 MiNiFi 的非常小的边缘设备一直到使用 NiFi 的大型数据中心.NiFi 的重点是视觉命令和控制、数据过滤、数据丰富、数据来源和安全性等功能，仅举几例.使用 NiFi，您无需编写代码并将其部署为作业，而是通过 UI 构建一个实时数据流，该数据流对每个操作生效.

Apache NiFi is a data flow tool that is focused on moving data between systems, all the way from very small edge devices with the use of MiNiFi, back to the larger data centers with NiFi. NiFi's focus is on capabilities like visual command and control, filtering of data, enrichment of data, data provenance, and security, just to name a few. With NiFi, you aren't writing code and deploying it as a job, you are building a living data flow through the UI that is taking effect with each action.

流处理平台通常专注于涉及流连接和窗口操作的计算.数据流工具通常是免费的，用于管理从源到处理平台的数据流.

Stream processing platforms are often focused on computations involving joins of streams and windowing operations. Where as a data flow tool is often complimentary and used to manage the flow of data from the sources to the processing platforms.

实际上 NiFi 和流处理系统之间有几个集成点……有 Flink、Spark、Storm 和 Apex 的组件可以从 NiFi 拉取数据，或将数据推回 NiFi.另一种常见的模式是使用 MiNiFi + NiFi 将数据导入 Apache Kafka，然后让流处理系统从 Kafka 消费.

There are actually several integration points between NiFi and stream processing systems... there are components for Flink, Spark, Storm, and Apex that can pull data from NiFi, or push data back to NiFi. Another common pattern would be to use MiNiFi + NiFi to get data into Apache Kafka, and then have the stream processing systems consume from Kafka.

这篇关于Apache Beam 和 Apache Nifi 的区别的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！