您将Apache Kafka用于什么? [英] What do you use Apache Kafka for?

问题描述

我想问一下我对Kafka的理解是否正确.

对于真正非常大的数据流，常规数据库是不够的，因此人们使用诸如Hadoop或Storm之类的东西.卡夫卡(Kafka)位于上述数据库的上方，并提供实时数据应去往的方向?

解决方案

我不这么认为.

Kafka 是邮件系统，它不位于数据库顶部.

您可以将 Kafka 与 ActiveMQ ， RabbitMQ 等消息传递系统进行比较.

摘自Apache文档

客户端和服务器之间的通信是通过简单，高性能，与语言无关的TCP协议完成的.

用例:

1. 消息传递: Kafka可以很好地替代传统的消息中介.在这个领域，Kafka与ActiveMQ或RabbitMQ等传统消息传递系统相当
2. 网站活动跟踪: Kafka最初的用例是能够将用户活动跟踪管道重建为一组实时的发布-订阅供稿
3. 指标:Kafka通常用于运营监控数据，其中涉及汇总来自分布式应用程序的统计信息，以生成运营数据的集中供稿
4. 日志汇总
5. 流处理
6. 事件源是一种应用程序设计样式，其中状态更改以按时间顺序的记录顺序记录.
7. 提交日志: Kafka可以用作分布式系统的一种外部提交日志.该日志有助于在节点之间复制数据，并充当故障节点恢复其数据的重新同步机制

I would like to ask if my understanding of Kafka is correct.

For really really big data stream, conventional database is not adequate so people use things such as Hadoop or Storm. Kafka sits on top of said databases and provide ...directions where the real time data should go?

解决方案

I don't think so.

Kafka is messaging system and it does not sit on top of database.

You can compare Kafka with messaging systems like ActiveMQ, RabbitMQ etc.

From Apache documentation page

Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.

Key takeaways:

1. Kafka maintains feeds of messages in categories called topics.
2. We'll call processes that publish messages to a Kafka topic producers.
3. We'll call processes that subscribe to topics and process the feed of published messages consumers..
4. Kafka is run as a cluster comprised of one or more servers each of which is called a broker.

Communication between the clients and the servers is done with a simple, high-performance, language agnostic TCP protocol.

Use Cases:

1. Messaging: Kafka works well as a replacement for a more traditional message broker. In this domain Kafka is comparable to traditional messaging systems such as ActiveMQ or RabbitMQ
2. Website Activity Tracking: The original use case for Kafka was to be able to rebuild a user activity tracking pipeline as a set of real-time publish-subscribe feeds
3. Metrics: Kafka is often used for operational monitoring data, which involves aggregating statistics from distributed applications to produce centralized feeds of operational data
4. Log Aggregation
5. Stream Processing
6. Event sourcing is a style of application design where state changes are logged as a time-ordered sequence of records.
7. Commit Log: Kafka can serve as a kind of external commit-log for a distributed system. The log helps replicate data between nodes and acts as a re-syncing mechanism for failed nodes to restore their data

这篇关于您将Apache Kafka用于什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！