直写RAM磁盘或文件系统的大规模缓存？ [英] write-through RAM disk, or massive caching of file system?

问题描述

我有一个程序是非常严重的文件系统，读取和写入一组工作文件。这些文件的大小是几千兆字节，但不能太大以至于不能放在RAM磁盘上。这个程序运行的机器通常是Ubuntu的Linux机器。

有没有办法将文件管理器配置成非常大的缓存，甚至是缓存写入他们之后又碰到了磁盘？

或者有没有办法创建一个写入实际磁盘的RAM磁盘？

ext3 ，尝试使用 data = writeback 甚至 async noatime 来减少文件系统活动。

在编程上，您可能还想通过内存映射执行磁盘访问（使用 mmap ）。这有点儿手工，但是对数据管理和优化提供了更多的控制权。

I have a program that is very heavily hitting the file system, reading and writing to a set of working files. The files are several gigabytes in size, but not so large as to not fit on a RAM-disk. The machines this program runs on are typically Ubuntu Linux boxes.

Is there a way to configure the file manager to have a very very large cache, and even to cache writes so they hit the disk later?

Or is there a way to create a RAM disk that writes-through to real disk?

解决方案

By default, Linux will use free RAM (almost all of it) to cache disk accesses, and will delay writes. The heuristics used by the kernel to decide the caching strategy are not perfect, but beating them in a specific situation is not easy. Also, on journalling filesystems (i.e. all the default filesystems nowadays), actual writes to the disk will be performed in a way which is resilient the crashes; this implies a bit of overhead. You may want to try to fiddle with filesystem options. E.g., for ext3, try mounting with data=writeback or even async (these options may improve filesystem performance, at the expense of reduced resilience towards crashes). Also, use noatime to reduce filesystem activity.

Programmatically, you might also want to perform disk accesses through memory mappings (with mmap). This is a bit hand-on, but it gives more control about data management and optimization.

这篇关于直写RAM磁盘或文件系统的大规模缓存？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！