使用内存映射文件读取大文件 [英] Reading huge files using Memory Mapped Files

问题描述

我看到许多文章建议不要将大型文件映射为mmap文件，这样虚拟地址空间将不会仅由mmap占用.

I see many articles suggesting not to map huge files as mmap files so the virtual address space won't be taken solely by the mmap.

在地址空间急剧增加的64位进程中，这将如何改变? 如果我需要随机访问一个文件，是否有理由不一次映射整个文件? (数十个GBs文件)

How does that change with 64 bit process where the address space dramatically increases? If I need to randomly access a file, is there a reason not to map the whole file at once? (dozens of GBs file)

推荐答案

在64位上，继续并映射文件.

On 64bit, go ahead and map the file.

根据Linux经验，需要考虑的一件事:如果访问确实是随机的，并且文件的大小比您预期在RAM中缓存的大得多(因此再次命中页面的机会很小)，那么它值得将MADV_RANDOM指定为 madvise 可停止稳定且毫无意义地交换命中文件页面其他实际有用的东西.虽然不知道 Windows等效API .

One thing to consider, based on Linux experience: if the access is truly random and the file is much bigger than you can expect to cache in RAM (so the chances of hitting a page again are slim) then it can be worth specifying MADV_RANDOM to madvise to stop the accumulation of hit file pages steadily and pointlessly swapping other actually useful stuff out. No idea what the windows equivalent API is though.

这篇关于使用内存映射文件读取大文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！