Amazon Kinesis 和保证订购 [英] Amazon Kinesis and guaranteed ordering

问题描述

亚马逊声称他们的 Kinesis 流媒体产品保证记录订购.

Amazon claims their Kinesis streaming product guarantees record ordering.

它提供记录的排序，以及以相同顺序读取和/或重放记录的能力 (...)

It provides ordering of records, as well as the ability to read and/or replay records in the same order (...)

Kinesis 由 Streams 组成，而 Streams 本身又由一个或多个 Shard 组成.记录存储在这些分片中.我们可以编写连接到分片的消费者应用程序，并按照记录的存储顺序读取/重放记录.

Kinesis is composed of Streams that are themselves composed of one or more Shards. Records are stored in these Shards. We can write consumer applications that connect to a Shard and read/replay records in the order they were stored.

但是 Kinesis 能否保证开箱即用地为 Stream 本身进行排序，而无需将排序逻辑推送给消费者?消费者如何从同一个流的多个分片中读取记录，确保记录的读取顺序与它们添加到流中的顺序相同?

But can Kinesis guarantee, out of the box, ordering for the Stream itself without pushing ordering logic to the consumers? How can a consumer read records from multiple Shards of the same Stream, making sure the records are read in the same order they were added to the Stream?

推荐答案

这似乎是不可能实现的.排序在分片级别得到保证，但不能在所有流中进行.

It seems this is not possible to achieve. Ordering is guaranteed on a shard level, but not across the all stream.

https://brandur.org/kinesis-order

回到我们最初的问题:我们如何保证所有记录的消费顺序与它们产生的顺序相同?这答案是我们不能，但我们不应该让那不幸现实太困扰我们了.一旦我们将流扩展到多个分片，我们没有任何机制可以用来保证记录在整个流中按顺序消耗；只在一个分片.

So back to our original question: how can we guarantee that all records are consumed in the same order in which they’re produced? The answer is that we can’t, but that we shouldn’t let that unfortunate reality bother us too much. Once we’ve scaled our stream to multiple shards, there’s no mechanism that we can use to guarantee that records are consumed in order across the whole stream; only within a single shard.

这篇关于Amazon Kinesis 和保证订购的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！