64 位大型 malloc [英] 64 bit large mallocs

问题描述

malloc() 失败的原因是什么，尤其是在 64 位中?

What are the reasons a malloc() would fail, especially in 64 bit?

我的具体问题是试图在 64 位系统上分配一个巨大的 10GB 内存块.这台机器有 12GB 的 RAM 和 32GB 的交换空间.是的，malloc 是极端的，但为什么会出现问题呢?这是在带有 Intel 和 MSFT 编译器的 Windows XP64 中.malloc 有时成功，有时不成功，大约 50%.8GB mallocs 总是有效，20GB mallocs 总是失败.如果 malloc 失败，重复请求将不起作用，除非我退出进程并再次启动一个新进程(这将有 50% 的成功率).没有其他大型应用程序正在运行.它甚至在重新启动后立即发生.

My specific problem is trying to malloc a huge 10GB chunk of RAM on a 64 bit system. The machine has 12GB of RAM, and 32 GB of swap. Yes, the malloc is extreme, but why would it be a problem? This is in Windows XP64 with both Intel and MSFT compilers. The malloc sometimes succeeds, sometimes doesn't, about 50%. 8GB mallocs always work, 20GB mallocs always fail. If a malloc fails, repeated requests won't work, unless I quit the process and start a fresh process again (which will then have the 50% shot at success). No other big apps are running. It happens even immediately after a fresh reboot.

如果您已经用完可用的 32(或 31)位地址空间，我可以想象 malloc 会在 32 位失败，这样就没有足够大的地址范围来分配给您的请求.

I could imagine a malloc failing in 32 bit if you have used up the 32 (or 31) bits of address space available, such that there's no address range large enough to assign to your request.

我也可以想象如果你的物理 RAM 和你的硬盘交换空间用完了，malloc 会失败.对我来说不是这样.

I could also imagine malloc failing if you have used up your physical RAM and your hard drive swap space. This isn't the case for me.

但是为什么 malloc 会失败?我想不出其他原因.

But why else could a malloc fail? I can't think of other reasons.

我对一般的 malloc 问题比我的具体示例更感兴趣，无论如何我可能会用内存映射文件替换它.失败的 malloc() 比其他任何事情都更像是一个难题……希望了解您的工具而不是对基础知识感到惊讶.

I'm more interested in the general malloc question than my specific example, which I'll likely replace with memory mapped files anyway. The failed malloc() is just more of a puzzle than anything else... that desire to understand your tools and not be surprised by the fundamentals.

推荐答案

malloc 尝试分配一个连续的内存范围，这最初将在实际内存中，这仅仅是由于交换内存的工作方式(至少据我所知).很可能您的操作系统有时无法找到连续的 10GB 内存块，并且仍将所有需要实际内存的进程同时留在 RAM 中(此时您的 malloc 将失败).

malloc tries to allocate a contiguous memory range, and this will initially be in real memory simply due to how swap memory works (at least as far as I remember). It could easily be that your OS sometimes can't find a contiguous block of 10gb of memory and still leave all the processes that require real memory in RAM at the same time (at which point your malloc will fail).

您实际上是否需要 10gb 的连续内存，或者您是否能够将存储类/结构包装在几个较小的块周围并以块的形式使用您的内存?这放宽了巨大的连续要求，并且还应该允许您的程序将交换文件用于较少使用的块.

Do you actually require 10gb of contiguous memory, or would you be able to wrap a storage class/struct around several smaller blocks and use your memory in chunks instead? This relaxes the huge contiguous requirement and should also allow your program to use the swap file for less used chunks.

这篇关于64 位大型 malloc的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！