使用`madvise`调零大内存映射 [英] Zero a large memory mapping with `madvise`

问题描述

我有以下问题:

我通过mmap和MAP_ANONYMOUS分配了大量的内存(多个GiB).该块包含一个较大的哈希图，该哈希图不时需要清零.并非在每个回合中都可以使用整个映射(并非每个页面都有错误)，所以memset并不是一个好主意-花费的时间太长.

I allocate a large chunk of memory (multiple GiB) via mmap with MAP_ANONYMOUS. That chunk holds a large hash map which needs to be zeroed every now and then. Not the entire mapping may be used in each round (not every page is faulted in), so memset is not a good idea - takes too long.

快速执行此操作的最佳策略是什么?

What is the best strategy to do this quickly?

将

madvise(ptr, length, MADV_DONTNEED);

保证我以后的访问都会提供新的空白页面吗?

guarantee me that any subsequent accesses provide new empty pages?

在Linux man madvise页面上:

From the Linux man madvise page:

此调用不会影响应用程序的语义( MADV_DONTNEED 除外)，但可能会影响其性能.内核可以随意忽略建议.

This call does not influence the semantics of the application (except in the case of MADV_DONTNEED), but may influence its performance. The kernel is free to ignore the advice.

...

MADV_DONTNEED

此范围内的页面的后续访问将成功，但是将导致从底层映射文件中重新加载内存内容(请参阅mmap(2))或按需按需填充零页面以进行映射而无需底层文件

Subsequent accesses of pages in this range will succeed, but will result either in reloading of the memory contents from the underlying mapped file (see mmap(2)) or zero-fill-on-demand pages for mappings without an underlying file.

...

当前的Linux实现(2.4.0)将此系统调用更多地视为命令而不是建议...

The current Linux implementation (2.4.0) views this system call more as a command than as advice ...

还是我必须munmap并重新映射该区域?

Or do I have to munmap and remap the region anew?

它必须在Linux上运行，并且理想情况下在OS X上具有相同的行为.

It has to work on Linux and ideally have the same behaviour on OS X.

推荐答案

对于您的问题，有一个非常容易解决的解决方案，可移植性很强:

There is a much easier solution to your problem that is fairly portable:

mmap(ptr, length, PROT_READ|PROT_WRITE, MAP_FIXED|MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);

由于出于任何特定于实现的原因而允许MAP_FIXED失败，因此，如果返回MAP_FAILED则退回到memset是可取的.

Since MAP_FIXED is permitted to fail for fairly arbitrary implementation-specific reasons, falling back to memset if it returns MAP_FAILED would be advisable.

这篇关于使用`madvise`调零大内存映射的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！