Cassandra Mem表内容 [英] Cassandra Mem table content

问题描述

cassandra yaml文件中有一个内存表堆大小配置。.说是2gb ...现在清理阈值是否为33％。.然后在675 mb内存表空间被占用之后..cassandra将刷新我的问题是cassandra会使用剩余的mem表空间1373 mb（2048-675）。

根据我的理解，内存空间中的时间点数据将不超过675 Mb，内存表数据增长到超过675 mb的时刻，最大的内存表被刷新到磁盘，内存表空间中的数据大小再次变为小于675 mb ...并且此过程继续...然后为什么我们需要分配2GB的mem表空间...这是什么原因... mem table dpace是否包含除mem table之外的任何内容...

解决方案

刷新不是立即进行的，它不会阻止其他写入的进入。它实际上为写入创建了一个新的活动内存表，并将前一个写入排队被冲洗到磁盘（在刷新前仍可用于读取）。因此，堆上使用的空间绝对可以超过您的阈值*空间。

此行为在旧版本上是不同的

因为Cassandra的大小实际上会阻塞写入，直到刷新完成（tpstats在FlushWriter下将其阻塞）。

刷新时，内存表可以继续增长，但有一个限制（ memtable_heap_space_in_mb 设置），该限制实际上会停止写入，以防止其旋转失控并导致OutOfMemory异常。此设置比可以立即分配并保留给内存表的内存块要大得多。

还要注意 memtable_cleanup_threshold 已过时：

默认计算是唯一合理的选择。

There is a mem table heap size config in cassandra yaml file..lets say it's 2gb...now if clean up threshold is 33%..then after 675 mb of mem table space is occupied..cassandra will flush the largest mem table to disk..My question is what cassandra does with the remaining mem table space that is 1373 mb(2048-675).

According to my understanding at any point of time data in memtable space will not be more than 675 Mb,the moment mem table data grows beyond 675 mb,largest memtable get flushed to disk and data size in mem table space again becomes less than 675 mb...and this process goes on...then why we need to assign 2GB of mem table space...what is the reason behind it...does mem table dpace contains any thing other than mem table ...

解决方案

Flushing is not instantaneous and it does not stop other writes from coming in. It essentially creates a new "active" memtable for the writes and puts the previous one on a queue to get flushed to disk (it can still be used for reads until flushed). So the space used on heap can most definitely exceed your threshold * space.

This behavior is different on older versions of Cassandra where it would actually block the writes until the flush completes (tpstats showed this as blocked under the FlushWriter, which is no longer possible).

Since the size of the memtables can continue to grow while flushing occurs, there is a cut off limit (the memtable_heap_space_in_mb setting) where it would actually stop writes to prevent it spinning out of control and causing OutOfMemory exceptions. This setting is more a limit that it can be grown to than a chunk of memory allocated immediately and reserved for the memtables.

Also note the memtable_cleanup_threshold is deprecated:

The default calculation is the only reasonable choice.

这篇关于Cassandra Mem表内容的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！