将Kafka用作(CQRS)事件存储.好主意? [英] Using Kafka as a (CQRS) Eventstore. Good idea?

问题描述

尽管我之前遇到过 Kafka ，但我最近才意识到，Kafka可能被用作( CQRS ，

Although I've come across Kafka before, I just recently realized Kafka may perhaps be used as (the basis of) a CQRS, eventstore.

Kafka支持的主要观点之一:

One of the main points that Kafka supports:

• 事件捕获/存储，当然是所有HA.
• Pub/sub体系结构
• 具有重播事件日志的功能，使新订户可以在事件发生后向系统注册.

诚然，我不是100％精通CQRS/事件源，但是这似乎与事件存储应有的程度非常接近.有趣的是:关于Kafka用作事件存储，我真的找不到太多，所以也许我错过了一些东西.

Admittedly I'm not 100% versed into CQRS / Event sourcing but this seems pretty close to what an eventstore should be. Funny thing is: I really can't find that much about Kafka being used as an eventstore, so perhaps I am missing something.

那么，Kafka缺少一个好的活动存储库吗?能行吗?用它生产吗?对洞察力，链接等感兴趣.

So, anything missing from Kafka for it to be a good eventstore? Would it work? Using it production? Interested in insight, links, etc.

基本上，系统状态是根据系统曾经收到的事务/事件保存的，而不仅仅是保存系统的当前状态/快照(通常这样做). (将其视为会计中的总帐:最终所有交易都达到最终状态).这允许进行各种有趣的事情，但请仔细阅读提供的链接.

Basically the state of the system is saved based on the transactions/events the system has ever received, instead of just saving the current state / snapshot of the system which is what is usually done. (Think of it as a General Ledger in Accounting: all transactions ultimately add up to the final state) This allows all kinds of cool things, but just read up on the links provided.

推荐答案

Kafka是一个消息传递系统，它与事件存储库有很多相似之处，但引述了它们的介绍:

Kafka is meant to be a messaging system which has many similarities to an event store however to quote their intro:

Kafka集群会保留所有已发布的消息-无论它们是否 已被消耗-一段可配置的时间段.例如，如果 保留期设置为两天，然后为 消息已发布，可供消费，之后便可以使用 将被丢弃以释放空间.卡夫卡的表现很有效 关于数据大小的常量，因此保留大量数据不是 问题.

The Kafka cluster retains all published messages—whether or not they have been consumed—for a configurable period of time. For example if the retention is set for two days, then for the two days after a message is published it is available for consumption, after which it will be discarded to free up space. Kafka's performance is effectively constant with respect to data size so retaining lots of data is not a problem.

因此，尽管可以无限期地保留消息，但是期望它们将被删除.这并不意味着您不能将其用作事件存储，但是最好使用其他东西.看看 EventStore 作为替代方案.

So while messages can potentially be retained indefinitely, the expectation is that they will be deleted. This doesn't mean you can't use this as an event store, but it may be better to use something else. Take a look at EventStore for an alternative.

Kafka文档:

事件源是一种应用程序设计样式，其中状态更改以时间顺序的记录序列记录下来. Kafka对大量存储的日志数据的支持使其成为使用这种样式构建的应用程序的绝佳后端.

Event sourcing is a style of application design where state changes are logged as a time-ordered sequence of records. Kafka's support for very large stored log data makes it an excellent backend for an application built in this style.

更新2

使用Kafka进行活动采购的一个问题是所需主题的数量.通常，在事件源中，每个实体(例如用户，产品等)都有事件流(主题).这样，可以通过重新应用流中的所有事件来重构实体的当前状态.每个Kafka主题由一个或多个分区组成，每个分区都存储为文件系统上的目录.随着znode数量的增加，ZooKeeper也将面临压力.

UPDATE 2

One concern with using Kafka for event sourcing is the number of required topics. Typically in event sourcing, there is a stream (topic) of events per entity (such as user, product, etc). This way, the current state of an entity can be reconstituted by re-applying all events in the stream. Each Kafka topic consists of one or more partitions and each partition is stored as a directory on the file system. There will also be pressure from ZooKeeper as the number of znodes increases.

这篇关于将Kafka用作(CQRS)事件存储.好主意?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！