Apache Kafka对比Apache Storm [英] Apache Kafka vs Apache Storm

问题描述

Apache Kafka：分布式消息传递系统

Apache Storm：实时消息处理

Apache Kafka: Distributed messaging system
Apache Storm: Real Time Message Processing

我们如何实时使用这两种技术数据管道来处理事件数据？

How we can use both technologies in a real-time data pipeline for processing event data?

就实时数据管道而言，在我看来两者都做得一样。我们如何在数据管道上同时使用这两种技术？

In terms of real time data pipeline both seems to me do the job identical. How can we use both the technologies on a data pipeline?

推荐答案

您将Apache Kafka用作可处理的分布式且健壮的队列高容量数据，使您能够将消息从一个端点传递到另一个端点。

You use Apache Kafka as a distributed and robust queue that can handle high volume data and enables you to pass messages from one end-point to another.

风暴不是一个队列。这是一个具有实时处理能力的系统，这意味着您可以并行执行对实时数据的各种处理。

Storm is not a queue. It is a system that has distributed real time processing abilities, meaning you can execute all kind of manipulations on real time data in parallel.

这些工具的通用流程（如我知道））如下：

The common flow of these tools (as I know it) goes as follows:

实时系统->卡夫卡->风暴-> NoSql-> BI（可选）

real-time-system --> Kafka --> Storm --> NoSql --> BI(optional)

因此，您可以使用实时应用程序处理大量数据，并将其发送到Kafka队列。 Storm从kafka提取数据并应用一些必需的操作。此时，您通常希望从此数据中获得一些好处，因此您可以将其发送到Nosql db进行其他BI计算，也可以从任何其他系统中查询此NoSql。

So you have your real time app handling high volume data, sends it to Kafka queue. Storm pulls the data from kafka and applies some required manipulation. At this point you usually like to get some benefits from this data, so you either send it to some Nosql db for additional BI calculations, or you could simply query this NoSql from any other system.

这篇关于Apache Kafka对比Apache Storm的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！