如何使用仅手动同步到磁盘的 mmap 获得检查点文件 [英] How to have a checkpoint file using mmap which is only synced to disk manually

问题描述

我需要最快的方法来定期将文件与内存同步.

I need the fastest way to periodically sync file with memory.

我想我想要一个 mmap 文件，它只能手动同步到磁盘.我不确定如何防止发生任何自动同步.

What I think I would like is to have an mmap'd file, which is only sync'd to disk manually. I'm not sure how to prevent any automatic syncing from happening.

除非在我手动指定的时间，否则无法修改该文件.关键是要有一个检查点文件，它将状态的快照保存在内存中.我想尽可能避免复制，因为这需要相当频繁地调用并且速度很重要.

The file cannot be modified except at the times I manually specify. The point is to have a checkpoint file which keeps a snapshot of the state in memory. I would like to avoid copying as much as possible, since this will be need to called fairly frequently and speed is important.

推荐答案

mmap 不能用于此目的.没有办法阻止数据写入磁盘.在实践中，使用 mlock() 使内存不可交换可能 有一个副作用是阻止它被写入磁盘，除非你要求它被写入，但是没有保证.当然，如果另一个进程打开该文件，它将看到缓存在内存中的副本(以及您的最新更改)，而不是物理磁盘上的副本.在许多方面，您应该做什么取决于您是尝试与其他进程同步，还是只是为了在崩溃或电源故障的情况下安全.

mmap can't be used for this purpose. There's no way to prevent data from being written to disk. In practice, using mlock() to make the memory unswappable might have a side effect of preventing it from getting written to disk except when you ask for it to be written, but there's no guarantee. Certainly if another process opens the file, it's going to see the copy cached in memory (with your latest changes), not the copy on physical disk. In many ways, what you should do depends on whether you're trying to do synchronization with other processes or just for safety in case of crash or power failure.

如果您的数据量很小，您可以尝试多种其他方法以原子同步到磁盘.一种方法是将整个数据集存储在一个文件名中并使用该名称创建一个空文件，然后删除旧文件.如果启动时存在 2 个文件(由于极不可能的崩溃时间)，请删除旧文件并从新文件恢复.如果您的数据大小小于文件系统块、页面大小或磁盘块，write() 可能也是原子的，但我不知道对此有任何保证立即生效.你必须做一些研究.

If your data size is small, you might try a number of other methods for atomic syncing to disk. One way is to store the entire dataset in a filename and create an empty file by that name, then delete the old file. If 2 files exist at startup (due to extremely unlikely crash time), delete the older one and resume from the newer one. write() may also be atomic if your data size is smaller than a filesystem block, page size, or disk block, but I don't know of any guarantee to that effect right off. You'd have to do some research.

另一种非常标准的方法，只要您的数据不是太大以至于磁盘无法容纳 2 个副本:只需使用临时名称创建第二个副本，然后 rename()它超过了旧的.rename() 总是原子的.除非您有理由不这样做，否则这可能是最好的方法.

Another very standard approach that works as long as your data isn't so big that 2 copies won't fit on disk: just create a second copy with a temporary name, then rename() it over top of the old one. rename() is always atomic. This is probably the best approach unless you have a reason not to do it that way.

这篇关于如何使用仅手动同步到磁盘的 mmap 获得检查点文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！