使用mmap保留地址空间的开销 [英] overhead of reserving address space using mmap

问题描述

我有一个程序通常使用大型数组，其中的内存是使用mmap

I have a program that routinely uses massive arrays, where the memory is allocated using mmap

是否有人知道在提交内存之前分配大量地址空间的典型开销，无论是使用MAP_NORESERVE进行分配还是使用稀疏文件来备份空间? It5令我震惊，mmap不能免费，因为它必须为分配的空间创建页表条目.在实施我正在考虑的算法之前，我想对此开销有所了解.

Does anyone know the typical overheads of allocating address space in large amounts before the memory is committed, either if allocating with MAP_NORESERVE or backing the space with a sparse file? It5 strikes me mmap can't be free since it must make page table entries for the allocated space. I want to have some idea of this overhead before implementing an algorithm I'm considering.

显然，答案将取决于平台，我对x64 linux，sparc solaris和sparc linux最感兴趣.我认为1mb页面的可用性使Sparc上的开销比x64少.

Obviously the answer is going to be platform dependent, im most interested in x64 linux, sparc solaris and sparc linux. I'm thinking that the availability of 1mb pages makes the overhead rather less on a sparc than x64.

推荐答案

mmap的开销取决于您使用它的方式.当您以适当的方式使用它时，通常可以忽略不计.

The overhead of mmap depends on the way you use it. And it is typically negligible when you use it in an appropriate way.

在linux内核中，mmap操作可以分为两部分:

In linux kernel, mmap operation can be divided into two parts:

1. 寻找可以保存映射的免费地址范围

1. look for a free address range that can hold the mapping

在地址空间(mm_struct)中创建/扩大vma结构

Create/enlarge vma struct in address space (mm_struct)

因此分配大量的内存使用量mmap不要介绍更多 开销比小.

So allocate large amount of memory use mmap do not introduce more overhead than small ones.

因此，您应该每次分配尽可能大的内存. (避免多次出现mmap的次数)

So you should allocate memory as larger as possible in each time. (avoid mutiple times of small mmap)

并且您可以显式提供起始地址(如果可能).这样可以节省内核寻找足够大的可用空间的时间.

如果您的应用程序是多线程程序. 您应避免同时调用mmap .这是因为地址空间受读写器锁保护，而mmap始终采用写入器锁.在这种情况下，mmap延迟将增加几个数量级.

If your application is an multi-threaded program. You should avoid concurrent calls to mmap. That is because the address space is protected by a reader-writer lock and the mmap always takes the writer lock. mmap latency will be orders of magnitude greater in this case.

此外，mmap仅创建映射，而不创建页面表.触摸页面时，将在页面错误处理程序中分配页面.页面错误处理程序将使用读取器锁来保护地址空间，并且还可能影响mmap性能.

Moreover, mmap only create the mapping but not the page table. Pages are allocated in the page fault handler when being touched. Page fault handler would take the reader lock that protects address space and can also affects mmap performance.

在这种情况下，您应始终尝试重用大型数组，而不是再次使用munmap并再次使用mmap.(避免页面错误)

In this case, you should always try to reuse your large array instead of munmap it and mmap again. (Avoid pagefaults)

这篇关于使用mmap保留地址空间的开销的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！