如何在 Cassandra 中快速增加计数器而不会过时 [英] how to rapidly increment counters in Cassandra w/o staleness

问题描述

我有一个卡桑德拉问题.您知道 Cassandra 如何更新/增加计数器吗?

I have a Cassandra question. Do you know how Cassandra does updates/increments of counters?

我想使用一个可以写入 cassandra 的风暴螺栓(来自 Storm-contrib repo on github 的 CassandraCounterBatchingBolt).但是，我不确定 incrementCounterColumn() 方法的某些实现是如何工作的……而且 cassandra 计数器也有一些限制(来自:http://wiki.apache.org/cassandra/Counters) 这使得它们对我的场景没用恕我直言:

I want to use a storm bolt (CassandraCounterBatchingBolt from storm-contrib repo on github) which writes into cassandra. However, I'm not sure how some of the implementation of the incrementCounterColumn() method works .. and there is also the limitations with cassandra counters (from: http://wiki.apache.org/cassandra/Counters) which makes them useless for my scenario IMHO:

• 如果写入意外失败(超时或与协调器节点的连接丢失)，客户端将不知道操作是否已执行.重试可能会导致 CASSANDRA-2495 计数过多.

• If a write fails unexpectedly (timeout or loss of connection to the coordinator node) the client will not know if the operation has been performed. A retry can result in an over count CASSANDRA-2495.

计数器删除本质上是有限的.例如，如果您非常快速地发出序列增量，删除，增量"，则删除可能会丢失

Counter removal is intrinsically limited. For instance, if you issue very quickly the sequence "increment, remove, increment" it is possible for the removal to be lost

无论如何，这是我的场景:
我更新同一个计数器的速度比更新传播到其他 Cassandra 节点的速度要快.

Anyway, here is my scenario:
I update the same counter faster than the updates propagate to other Cassandra nodes.

示例:
假设我有 3 个 cassandra 节点.每个节点上的计数器都是 0.
节点1:0、节点2:0、节点3:0

Example:
Say I have 3 cassandra nodes. The counters on each of these nodes are 0.
Node1:0, node2:0, node3:0

一个增量来了:5 -> Node1:0, node2:0, node3:0

An increment comes: 5 -> Node1:0, node2:0, node3:0

增量从节点 2 开始——仍然需要传播到节点 1 和节点 3
节点1:0、节点2:5、节点3:0

Increment starts at node 2 – still needs to propagate to node1 and node3
Node1:0, node2:5, node3:0

与此同时，另一个增量在前一个增量之前到达
传播:3 -> Node1:0, node2:5, node3:0

In the meantime, another increment arrives before previous increment
is propagated: 3 -> Node1:0, node2:5, node3:0

假设 3 个节点与 5 个节点在不同的节点开始，我们有:
节点1:3、节点2:5、节点3:0

Assuming 3 starts at a different node than where 5 started we have:
Node1:3, node2:5, node3:0

现在，如果 3 作为增量而不是作为新值传播到其他节点(和 5 相同)然后最终节点都等于 8，这就是我想要的.

Now if 3 gets propagated to the other nodes AS AN INCREMENT and not as a new value (and the same for 5) then eventually the nodes would all equal 8 and this is what I want.

如果 3 覆盖 5(因为它有一个较晚的时间戳)这是有问题的——不是我想要的.

If 3 overwrites 5 (because it has a later timestamp) this is problematic – not what I want.

您知道 Cassandra 如何处理这些更新/增量吗?

请注意，在写入之前的读取仍然容易受到相同的问题的影响，具体取决于读取是从哪个副本节点执行的(如果传播不远，Quorum 仍然可能失败)

Note, that a read before a write is still susceptible to the same problem depending from which replica node the read executes (Quorum can still fail if propagation is not far along)

我也在想，也许在我的风暴螺栓和 Cassandra 中放置一个缓存可能会解决这个问题，但这是另一个故事.

I'm also thinking that maybe putting a cache b/w my storm bolt and Cassandra might solve this issue but that's a story for another time.

推荐答案

C* 中的计数器具有复杂的内部表示，可以避免在无领导分布式系统中计算事物的大多数(但不是全部)问题.我喜欢将它们视为分片计数器.一个计数器由许多由主机 ID 和版本号标识的子计数器组成.接收到计数器操作的主机只会增加自己的子计数器，也会增加版本.然后它将其整个计数器状态复制到其他副本，这些副本将其与其状态合并.当计数器被读取时，处理读取操作的节点通过总结来自每个主机的计数的总和来确定计数器值.

Counters in C* have a complex internal representation that avoids most (but not all) problems of counting things in a leaderless distributed system. I like to think of them as sharded counters. A counter consists of a number of sub-counters identified by host ID and a version number. The host that receives the counter operation increments only its own sub-counter, and also increments the version. It then replicates its whole counter state to the other replicas, which merge it with their states. When the counter is read the node handling the read operation determines the counter value by summing up the total of the counts from each host.

在每个节点上，计数器增量就像 Cassandra 中的其他所有内容一样，只是一次写入.增量写入memtable，本地值在读取时通过合并memtable和所有SSTable的所有增量来确定.

On each node a counter increment is just like everything else in Cassandra, just a write. The increment is written to the memtable, and the local value is determined at read time by merging all of the increments from the memtable and all SSTables.

当我说您不必担心计数器递增速度超过 Cassandra 处理能力时，我希望我的解释能帮助您相信我.由于每个节点都保留自己的计数器，并且从不复制增量操作，因此不可能像读取-修改-写入场景那样引入竞争条件导致计数丢失.如果 Cassandra 接受写入，您几乎可以保证它会计数.

I hope that explanation helps you believe me when I say that you don't have to worry about incrementing counters faster than Cassandra can handle. Since each node keeps its own counter, and never replicates increment operations, there is no possibility of counts getting lost by race conditions like a read-modify-write scenario would introduce. If Cassandra accepts the write, your're pretty much guaranteed that it will count.

但是，您不能保证计数始终显示正确，除非.如果将增量写入一个节点，但随后从另一个节点读取计数器值，则不能保证增量已被复制，您还必须考虑在网络分区期间会发生什么.这或多或少与 Cassandra 中的任何写入相同，它具有最终一致性的性质，并且取决于您用于操作的一致性级别.

What you're not guaranteed, though, is that the count will appear correct at all times unless. If an increment is written to one node but the counter value read from another just after, there is not guarantee that the increment has been replicated, and you also have to consider what would happen during a network partition. This more or less the same with any write in Cassandra, it's in its eventually consistent nature, and it depends on which consistency levels you used for the operations.

还存在丢失确认的可能性.如果您在获得响应之前执行增量并断开与 Cassandra 的连接，您将无法知道您的写入是否得到了响应.当您恢复连接时，您也无法判断，因为在增加之前您不知道计数是多少.对于选择可用性而非一致性的系统而言，这是一个固有的问题，并且您要为许多其他好处付出代价.

There is also the possibility of a lost acknowledgement. If you do an increment and loose the connection to Cassandra before you can get the response back you can't know whether or not your write got though. And when you get the connection back you can't tell either, since you don't know what the count was before you incremented. This is an inherent problem with systems that choose availability over consistency, and the price you pay for many of the other benefits.

最后，快速删除、增量、删除的问题是真实存在的，您应该避免.问题是增量操作本质上会重新激活列，如果这些操作彼此足够接近，它们可能会获得相同的时间戳.Cassandra 严格执行最后写入，并根据操作的时间戳确定最后一个.如果两个操作具有相同的时间戳，则较大"的一个获胜，这意味着按照严格的字节顺序排序的那个.这是真实的，但我不会太担心，除非您对相同的值进行非常快速的写入和删除(这可能是您的数据模型中的错误).

Finally, the issue of rapid remove, increment, removes are real, and something you should avoid. The problem is that the increment operation will essentially resurrect the column, and if these operations come close enough to each other they might get the same timestamp. Cassandra is strictly last-write-wins and determines last based on the timestamp of the operation. If two operations have the same time stamp, the "greater" one wins, which means the one which sorts after in a strict byte order. It's real, but I wouldn't worry too much about it unless you're doing very rapid writes and deletes to the same value (which is probably a fault in your data model).

这是 Cassandra 计数器内部结构的很好指南:http://www.datastax.com/wp-content/uploads/2011/07/cassandra_sf_counters.pdf

Here's a good guide to the internals of Cassandra's counters: http://www.datastax.com/wp-content/uploads/2011/07/cassandra_sf_counters.pdf

这篇关于如何在 Cassandra 中快速增加计数器而不会过时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！