流处理和消息处理的区别 [英] Difference between stream processing and message processing

问题描述

流处理和传统消息处理的基本区别是什么?正如人们所说，kafka 是流处理的不错选择，但本质上 kafka 是一个类似于 ActivMQ、RabbitMQ 等的消息传递框架.

What is the basic difference between stream processing and traditional message processing? As people say that kafka is good choice for stream processing but essentially kafka is a messaging framework similar to ActivMQ, RabbitMQ etc.

为什么我们通常不说 ActiveMQ 也适用于流处理.

Why do we generally not say that ActiveMQ is good for stream processing as well.

消费者消费消息的速度是否决定了它是否是流?

Is it the speed at which messages are consumed by the consumer determines if it is a stream?

推荐答案

在传统的消息处理中，您对消息应用简单的计算——在大多数情况下，每条消息单独计算.

In traditional message processing, you apply simple computations on the messages -- in most cases individually per message.

在流处理中，您可以同时对多个输入流和多个记录(即消息)应用复杂的操作(如聚合和连接).

In stream processing, you apply complex operations on multiple input streams and multiple records (ie, messages) at the same time (like aggregations and joins).

此外，传统的消息传递系统无法回到过去"——即在消息发送给所有订阅的消费者后自动删除消息.相比之下，Kafka 使用基于拉取的模型(即，消费者从 Kafka 中拉取数据)时会在可配置的时间内保留消息.这允许消费者倒带"并多次使用消息——或者如果您添加一个新的消费者，它可以读取完整的历史记录.这使得流处理成为可能，因为它允许更复杂的应用程序.此外，流处理不一定是实时处理——它是关于处理无限输入流(与应用于有限输入的批处理相反).

Furthermore, traditional messaging system cannot go "back in time" -- ie, the automatically delete messages after they got delivered to all subscribed consumers. In contrast, Kafka keeps the messages as it uses a pull based model (ie, consumer pull data out of Kafka) for a configurable amount of time. This allows consumers to "rewind" and consume messages multiple times -- or if you add a new consumer, it can read the complete history. This makes stream processing possible, because it allows for more complex applications. Furthermore, stream processing is not necessarily about real-time processing -- it's about processing infinite input stream (in contrast to batch processing that is applied to finite inputs).

而且 Kafka 提供了 Kafka Connect 和 Streams API——因此它是一个流处理平台，而不仅仅是一个消息传递/发布-订阅系统(即使它在核心中使用它).

And Kafka offers Kafka Connect and Streams API -- so it is a stream processing platform and not just a messaging/pub-sub system (even if it uses this in it's core).

这篇关于流处理和消息处理的区别的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！