具有低磁盘空间的分层压缩策略 [英] Leveled Compaction Strategy with low disk space

问题描述

我们有具有Leveled压缩策略的Cassandra 1.1.1服务器。

系统工作，以便有读取和删除操作。每半年我们在新数据进入时删除大约一半的数据。有时会发生磁盘使用率上升到75％，而我们知道实际数据占据了墓碑占据的大约40-50％的其他空间。为了避免磁盘溢出，我们通过将所有SSTables删除为0级来强制压缩表。为此，我们删除.json清单文件并重新启动Cassandra节点。 （gc_grace选项没有帮助，因为只有在级别填满后才开始压缩）

从Cassandra 2.0开始，清单文件已移到sstable文件本身： https://issues.apache.org/jira/browse/CASSANDRA-4872

我们正在考虑迁移到Cassandra 2.x，而我们恐怕我们不会有这样的可能性，迫使进行压缩。

我的问题是：我们如何实现我们的表有磁盘空间限制例如150GB？ （当超过限制时，它会自动触发压缩）。问题主要是关于Cassandra 2.x.

解决方案

似乎我自己找到了答案。 / p>

• 有从2.x版本开始的工具sstablelevelreset，它与删除清单文件类似的级别重置。该工具位于Cassandra分发的tools目录中，例如




• 从Cassandra 1.2开始（ https://issues.apache.org/jira/browse/CASSANDRA-4234 ）有Leveled Compaction策略的逻辑删除支持，支持tombstone_threshold选项。它可以在表中设置墓碑的最大比例。

We have Cassandra 1.1.1 servers with Leveled Compaction Strategy.

The system works so that there are read and delete operations. Every half a year we delete approximately half of the data while new data comes in. Sometimes it happens that disk usage goes up to 75% while we know that real data take about 40-50% other space is occupied by tombstones. To avoid disk overflow we force compaction of our tables by dropping all SSTables to Level 0. For that we remove .json manifest file and restart Cassandra node. (gc_grace option does not help since compaction starts only after level is filled)

Starting from Cassandra 2.0 the manifest file was moved to sstable file itself: https://issues.apache.org/jira/browse/CASSANDRA-4872

We are considering migration to Cassandra 2.x while we afraid we won't have such a possibility as forcing leveled compaction any more.

My question is: how could we achieve that our table has a disk space limit e.g. 150GB? (When the limit is exceeded it triggers compaction automatically). The question is mostly about Cassandra 2.x. While any alternative solutions for Cassandra 1.1.1 are also welcome.

解决方案

It seems like I've found the answers myself.

• There is tool sstablelevelreset starting from 2.x version which does similar level reset as deletion of manifest file. The tool is located in tools directory of Cassandra distribution e.g. apache-cassandra-2.1.2/tools/bin/sstablelevelreset.

• Starting from Cassandra 1.2 (https://issues.apache.org/jira/browse/CASSANDRA-4234) there is tombstone removal support for Leveled Compaction Strategy which supports tombstone_threshold option. It gives the possibility of setting maximal ratio of tombstones in a table.

这篇关于具有低磁盘空间的分层压缩策略的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！