SSD的低延迟键值存储 [英] Low-latency Key-Value Store for SSD

问题描述

我们正在开发一个支持SSD的键值解决方案，它具有以下属性：

• 吞吐量：10000 TPS; 50/50 put / gets;


• 延迟：1ms平均，99.9百分位数10ms


• 数据量：约10亿个值， 64位密钥;随机访问，20％的数据适合RAM

我们在商用SSD上尝试KyotoCabinet，LevelDB和RethinkDB， ext3 / xfs文件系统;使用。

总结是，随着磨损均衡开销的增加，SSD写性能会随着时间而恶化。随着驱动器上的可用页面数量减少，NAND控制器必须开始碎片整理页面，这有助于延迟。 NAND还必须建立LBA以阻塞映射以跟踪跨各个NAND块的数据的随机分布。随着这个地图的增长，地图上的操作（插入，删除）会变得更慢。

你不能解决低级别的HW问题SW方法，您将需要向上移动到企业级SSD或放宽您的延迟要求。

We are working on a SSD-backed key-value solution with the following properties:

• Throughput: 10000 TPS; 50/50 puts/gets;
• Latency: 1ms average, 99.9th percentile 10ms
• Data volume: ~1 billion values, ~150 bytes each; 64-bit keys; random access, 20% of data fits RAM

We tried KyotoCabinet, LevelDB, and RethinkDB on commodity SSDs, with different Linux IO schedulers, ext3/xfs file systems; made a number of tests using Rebench; and found that in all cases:

• Read-only throughput/latency are very good
• Write/update-only throughout is moderate, but there are many high-latency outliers
• Mixed read/write workload causes catastrophic oscillation in throughput/latency even in case of direct access to the block device (bypassing the file system)

The picture below illustrates such behavior for KyotoCabinet (horizontal axis is time, three periods are clearly visible - read-only, mixed, update only).

The question is: is it possible to achieve low latency for described SLAs using SSDs and what key-value stores are recommended?

解决方案

Highly variant write latency is a common attribute of SSDs (especially consumer models). There is a pretty good explanation of why in this AnandTech review .

Summary is that the SSD write performance worsens overtime as the wear leveling overhead increases. As the number of free pages on the drive decreases the NAND controller must start defragmenting pages, which contributes to latency. The NAND also must build an LBA to block map to track the random distribution of data across various NAND blocks. As this map grows, operations on the map (inserts, deletions) will get slower.

You aren't going to be able to solve a low level HW issue with a SW approach, you are going to need to either move up to an enterprise level SSD or relax your latency requirements.

这篇关于SSD的低延迟键值存储的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！