Apache Kafka 与 Apache Storm [英] Apache Kafka vs Apache Storm

问题描述

Apache Kafka:分布式消息系统
Apache Storm:实时消息处理

Apache Kafka: Distributed messaging system
Apache Storm: Real Time Message Processing

我们如何在实时数据管道中使用这两种技术来处理事件数据?

How we can use both technologies in a real-time data pipeline for processing event data?

就实时数据管道而言，在我看来两者的工作完全相同.我们如何在数据管道上使用这两种技术?

In terms of real time data pipeline both seems to me do the job identical. How can we use both the technologies on a data pipeline?

推荐答案

您将 Apache Kafka 用作一个分布式且健壮的队列，它可以处理大量数据并使您能够将消息从一个端点传递到另一个端点.

You use Apache Kafka as a distributed and robust queue that can handle high volume data and enables you to pass messages from one end-point to another.

Storm 不是队列.这是一个具有分布式实时处理能力的系统，这意味着您可以并行地对实时数据执行各种操作.

Storm is not a queue. It is a system that has distributed real time processing abilities, meaning you can execute all kind of manipulations on real time data in parallel.

这些工具的通用流程(据我所知)如下:

The common flow of these tools (as I know it) goes as follows:

实时系统 --> Kafka --> Storm --> NoSql --> BI(可选)

real-time-system --> Kafka --> Storm --> NoSql --> BI(optional)

因此，您的实时应用程序可以处理大量数据，并将其发送到 Kafka 队列.Storm 从 kafka 中提取数据并应用一些必要的操作.此时，您通常希望从这些数据中获得一些好处，因此您可以将其发送到某个 Nosql 数据库以进行额外的 BI 计算，或者您可以简单地从任何其他系统查询此 NoSql.

So you have your real time app handling high volume data, sends it to Kafka queue. Storm pulls the data from kafka and applies some required manipulation. At this point you usually like to get some benefits from this data, so you either send it to some Nosql db for additional BI calculations, or you could simply query this NoSql from any other system.

这篇关于Apache Kafka 与 Apache Storm的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！