内存映射文件和单个块的原子写入 [英] Memory Mapped files and atomic writes of single blocks

问题描述

如果我使用普通的IO API读写单个文件，则可以确保每个块的写入都是原子的.也就是说，如果我的写入仅修改单个块，则操作系统将保证写入整个块，或者完全不写入任何内容.

If I read and write a single file using normal IO APIs, writes are guaranteed to be atomic on a per-block basis. That is, if my write only modifies a single block, the operating system guarantees that either the whole block is written, or nothing at all.

如何在内存映射文件上实现相同的效果?

How do I achieve the same effect on a memory mapped file?

内存映射文件只是字节数组，因此，如果我修改字节数组，则操作系统无法得知何时考虑写入完成"，因此它可能(即使不太可能)换出内存就在我的块编写操作的中间，实际上我写了半个块.

Memory mapped files are simply byte arrays, so if I modify the byte array, the operating system has no way of knowing when I consider a write "done", so it might (even if that is unlikely) swap out the memory just in the middle of my block-writing operation, and in effect I write half a block.

在写入文件时，我需要某种输入/离开关键部分"，或某种将文件页面固定"到内存中的方法.是否存在类似的东西?如果是这样，则可以跨普通的POSIX系统进行移植吗? Windows?

I'd need some sort of a "enter/leave critical section", or some method of "pinning" the page of a file into memory while I'm writing to it. Does something like that exist? If so, is that portable across common POSIX systems & Windows?

推荐答案

保持新闻似乎是唯一的方法.我不知道这对写入同一个文件的多个应用程序如何起作用. Cassandra项目有一篇好文章，内容涉及如何提高期刊的绩效.关键是要确保日志只记录积极动作(我的第一种方法是将每次写入的前映像写入日志，以允许您回滚，但是它过多了复杂).

The technique of keeping a journal seems to be the only way. I don't know how this works with multiple apps writing to the same file. The Cassandra project has a good article on how to get performance with a journal. The key thing is to make sure of, is that the journal only records positive actions (my first approach was to write the pre-image of each write to the journal allowing you to rollback, but it got overly complicated).

因此，基本上，您的内存映射文件的标头中有一个transactionId，如果您的标头适合一个块，您就知道它不会损坏，尽管许多人似乎用校验和两次将其写入:[header[cksum]] [header[cksum]] .如果第一个校验和失败，请使用第二个.

So basically your memory-mapped file has a transactionId in the header, if your header fits into one block you know it won't get corrupted, though many people seem to write it twice with a checksum: [header[cksum]] [header[cksum]]. If the first checksum fails, use the second.

日记看起来像这样:

[beginTxn[txnid]] [offset, length, data...] [commitTxn[txnid]]

您只需继续添加日记记录，直到记录变得太大为止，然后将其滚动到某个位置.启动程序时，请检查文件的事务ID是否在日志的最后一个事务ID中；否则，请回放日志中的所有事务以进行同步.

You just keep appending journal records until it gets too big, then roll it over at some point. When you startup your program you check to see if the transaction id for the file is at the last transaction id of the journal -- if not you play back all the transactions in the journal to sync up.

这篇关于内存映射文件和单个块的原子写入的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！