文件映射引起的文件碎片 [英] file fragmentation caused by file mapping

问题描述

我有一个程序(用c编写)，该程序可以同时创建200多个文件，并通过使用mmap将它们映射到程序地址空间中同时写入所有文件.一切正常，但是当我备份目录时，由同一程序一次创建和写入的文件从同一磁盘上复制下来的速度要比映射并写入几个字节的文件快得多(5-10倍).一个时间.我只能想象这是由于某种文件碎片造成的，尽管我认为这在ufs文件系统上甚至是不可能的.

I have a program (written in c) that creates 200+ files simultaneously and writes to them all simultaneously by using mmap to map them into the programs address space. Everything works fine, but when I come to back up the directory, files that are created and written in one go by the same program copy off the same disk much faster (5-10x) than those that are mapped and written a few bytes at a time. I can only imagine that this is due to some kind of file fragmentation, though I didn't think this was even possible on a ufs file system.

有人对以下内容有任何想法吗?

Does anyone have any ideas on:

1)尝试解决此问题甚至是个好主意，因为在正常使用下，文件可能会以与写入文件时相同的顺序回读?

1) is it even a good idea to try to solve this, given under normal use the files will likely be read back in the same order that they were written in?

2)如果有一个好的解决方案，该怎么办?

2) if there is a good solution to this, what to do about it?

磁盘的文件系统为ufs

The file system of the disk is ufs

推荐答案

最终我在Free BSD手册页中找到了

Eventually I found in the Free BSD man pages

警告！使用ftruncate(2)扩展文件，因此 创建一个大孔，然后通过修改共享的mmap()填充孔会导致严重的文件碎片- 精神.为了避免这种碎片，您 应始终通过以下方式预分配文件的后备存储 将零写入()到新扩展的区域中 通过mmap()修改区域.碎片问题对MAP_NOSYNC尤其敏感 页面，因为页面可能会刷新到磁盘中 完全随机的顺序."

"WARNING! Extending a file with ftruncate(2), thus creating a big hole, and then filling the hole by modifying a shared mmap() can lead to severe file frag- mentation. In order to avoid such fragmentation you should always pre-allocate the file's backing store by write()ing zero's into the newly extended area prior to modifying the area via your mmap(). The fragmentation problem is especially sensitive to MAP_NOSYNC pages, because pages may be flushed to disk in a totally random order."

我得出的结论是，尽管Linux或Solaris手册页没有这样的警告，但是问题必须同样存在.该怎么办是另一个问题-如果文件的最终大小在开始时未知，则在整个文件上写入零不是一个好主意.可能强迫使用大页面可能会有所帮助，但是这种性质的任何解决方案都将针对特定平台.

I have come to the conclusion that although the Linux or Solaris man pages have no such warning, the problem must exist just the same. What to do about it is another question - writing zeros over the whole file is not a good idea if the final size of the files are unknown at the beginning. Possibly forcing the use of large pages may help, but any solution of this nature is going to be platform specific.

这篇关于文件映射引起的文件碎片的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！