madvise(___, ___, MADV_DONTNEED) 是否指示操作系统懒惰地写入磁盘? [英] Does madvise(___, ___, MADV_DONTNEED) instruct the OS to lazily write to disk?

问题描述

假设我想对一个可能非常大的文件执行顺序写入.

Hypothetically, suppose I want to perform sequential writing to a potentially very large file.

如果我 mmap() 一个巨大的区域并在整个区域上使用 madvise(MADV_SEQUENTIAL)，那么我可以以相对有效的方式写入内存.我已经开始工作了.

If I mmap() a gigantic region and madvise(MADV_SEQUENTIAL) on that entire region, then I can write to the memory in a relatively efficient manner. This I have gotten to work just fine.

现在，为了在我编写时释放各种操作系统资源，我偶尔会在已经写入的小块内存上执行 munmap().我担心 munmap() 和 msync() 会阻塞我的线程，等待数据物理提交到磁盘.我根本不能让我的作家慢下来，所以我需要另辟蹊径.

Now, in order to free up various OS resources as I am writing, I occasionally perform a munmap() on small chunks of memory that have already been written to. My concern is that munmap() and msync()will block my thread, waiting for the data to be physically committed to disk. I cannot slow down my writer at all, so I need to find another way.

在已经写入的小内存块上使用 madvise(MADV_DONTNEED) 会更好吗?我想告诉操作系统懒惰地将该内存写入磁盘，而不是阻塞我的调用线程.

Would it be better to use madvise(MADV_DONTNEED) on the small, already-written chunk of memory? I want to tell the OS to write that memory to disk lazily, and not to block my calling thread.

madvise() 上的 manpage 有这样的说法，这是相当模棱两可的:

The manpage on madvise() has this to say, which is rather ambiguous:

MADV_DONTNEED
Do  not expect access in the near future.  (For the time being, the 
application is finished with the given range, so the kernel can free
resources associated with it.)  Subsequent accesses of pages in this
range will succeed, but will result either in re-loading  of the memory
contents from the underlying mapped file (see mmap(2)) or
zero-fill-on-demand pages for mappings without an underlying file.

推荐答案

否！

为了您自己的利益，请远离MADV_DONTNEED.Linux 不会将此作为提示在写回页面后将其丢弃，而是立即将其丢弃.这不被视为错误，而是经过深思熟虑的决定.

No!

For your own good, stay away from MADV_DONTNEED. Linux will not take this as a hint to throw pages away after writing them back, but to throw them away immediately. This is not considered a bug, but a deliberate decision.

具有讽刺意味的是，非破坏性 MADV_DONTNEED 的功能已经由 msync(MS_INVALIDATE|MS_ASYNC)、MS_ASYNC 提供另一方面不启动 I/O(事实上，它什么都不做，按照脏页写回工作正常的推理)，fsync 总是阻塞，并且 sync_file_range 可能会阻止，如果您超出某个模糊的限制并且被文档视为极其危险"，无论这意味着什么.

Ironically, the reasoning is that the functionality of a non-destructive MADV_DONTNEED is already given by msync(MS_INVALIDATE|MS_ASYNC), MS_ASYNC on the other hand does not start I/O (in fact, it does nothing at all, following the reasoning that dirty page writeback works fine anyway), fsync always blocks, and sync_file_range may block if you exceed some obscure limit and is considered "extremely dangerous" by the documentation, whatever that means.

无论哪种方式，您都必须msync(MS_SYNC)，或fsync(均阻塞)，或sync_file_range(可能阻塞)后跟fsync，否则MADV_DONTNEED将会丢失数据.如果您无法承受可能的阻塞，那么遗憾的是，您别无选择，只能在另一个线程中执行此操作.

Either way, you must msync(MS_SYNC), or fsync (both blocking), or sync_file_range (possibly blocking) followed by fsync, or you will lose data with MADV_DONTNEED. If you cannot afford to possibly block, you have no choice, sadly, but to do this in another thread.

这篇关于madvise(___, ___, MADV_DONTNEED) 是否指示操作系统懒惰地写入磁盘?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！