madvise(___，___，MADV_DONTNEED)是否指示OS延迟写入磁盘? [英] Does madvise(___, ___, MADV_DONTNEED) instruct the OS to lazily write to disk?

问题描述

假设，我想对可能非常大的文件执行顺序写入.

Hypothetically, suppose I want to perform sequential writing to a potentially very large file.

如果我mmap()一个巨大的区域，并在整个区域上使用madvise(MAD​​V_SEQUENTIAL)，那么我可以以一种相对有效的方式写入内存.我已经可以正常工作了.

If I mmap() a gigantic region and madvise(MADV_SEQUENTIAL) on that entire region, then I can write to the memory in a relatively efficient manner. This I have gotten to work just fine.

现在，为了释放我正在编写的各种OS资源，我有时会在已经写入的小内存块上执行munmap().我担心的是munmap()和msync()会阻塞我的线程，等待将数据物理提交到磁盘.我根本不能放慢作家的脚步，所以我需要找到另一种方法.

Now, in order to free up various OS resources as I am writing, I occasionally perform a munmap() on small chunks of memory that have already been written to. My concern is that munmap() and msync()will block my thread, waiting for the data to be physically committed to disk. I cannot slow down my writer at all, so I need to find another way.

在已经写入的较小的内存块上使用madvise(MAD​​V_DONTNEED)会更好吗?我想告诉操作系统将内存延迟地写入磁盘，而不要阻塞我的调用线程.

Would it be better to use madvise(MADV_DONTNEED) on the small, already-written chunk of memory? I want to tell the OS to write that memory to disk lazily, and not to block my calling thread.

madvise()的联机帮助页上有这么一句话，这很含糊:

The manpage on madvise() has this to say, which is rather ambiguous:

MADV_DONTNEED
Do  not expect access in the near future.  (For the time being, the 
application is finished with the given range, so the kernel can free
resources associated with it.)  Subsequent accesses of pages in this
range will succeed, but will result either in re-loading  of the memory
contents from the underlying mapped file (see mmap(2)) or
zero-fill-on-demand pages for mappings without an underlying file.

推荐答案

否！

为了您自己的利益，请远离MADV_DONTNEED. Linux将不以此为提示将页面写回后将其丢弃，但立即将其丢弃.这不是错误，而是经过深思熟虑的决定.

No!

For your own good, stay away from MADV_DONTNEED. Linux will not take this as a hint to throw pages away after writing them back, but to throw them away immediately. This is not considered a bug, but a deliberate decision.

具有讽刺意味的是，理由是msync(MS_INVALIDATE|MS_ASYNC)已经赋予了非破坏性MADV_DONTNEED功能，而另一方面MS_ASYNC却没有启动I/O(实际上，它什么都不做，遵循脏页写回仍然可以正常工作的理由)，fsync始终会阻塞，而sync_file_range 可能会阻塞，如果您超出了一些晦涩的限制，并且被文档视为极度危险"，无论如何手段.

Ironically, the reasoning is that the functionality of a non-destructive MADV_DONTNEED is already given by msync(MS_INVALIDATE|MS_ASYNC), MS_ASYNC on the other hand does not start I/O (in fact, it does nothing at all, following the reasoning that dirty page writeback works fine anyway), fsync always blocks, and sync_file_range may block if you exceed some obscure limit and is considered "extremely dangerous" by the documentation, whatever that means.

无论哪种方式，您都必须msync(MS_SYNC)或fsync(两个都被阻止)或sync_file_range(可能是被阻止)后跟fsync，否则您将丢失.如果您负担不起阻塞的费用，那么遗憾的是，您别无选择，只能在另一个线程中进行此操作.

Either way, you must msync(MS_SYNC), or fsync (both blocking), or sync_file_range (possibly blocking) followed by fsync, or you will lose data with MADV_DONTNEED. If you cannot afford to possibly block, you have no choice, sadly, but to do this in another thread.

这篇关于madvise(___，___，MADV_DONTNEED)是否指示OS延迟写入磁盘?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！