在 Mac OS X 上的 64 位进程中，中等大小的内存分配如何失败? [英] How can a moderately sized memory allocation fail in a 64 bit process on Mac OS X?

问题描述

我正在构建一个相册布局应用程序.该应用程序经常将 JPEG 图像解压缩到内存位图缓冲区中.图片大小限制为 100 兆像素(但通常不超过 15 兆像素).

有时这些缓冲区的内存分配会失败:[[NSMutableData alloc] initWithLength:] 返回 nil.这似乎发生在系统可用物理内存接近零的情况下.

我对Mac OS X 中的虚拟内存系统 是在 64 位进程中分配虚拟(原文如此)不会失败.有 16 艾字节的地址空间，我试图一次最多分配 400 兆字节.理论上，我可以分配 400 亿个这样的缓冲区，而不会达到可用地址空间的硬限制.当然，实际限制会阻止这种情况，因为交换空间受引导卷大小的限制.实际上，我只进行了很少的分配(不到十次).

我不明白的是分配失败的事实，无论此时物理内存有多低.我认为——只要还有交换空间剩余——内存分配就不会失败(因为此时页面甚至没有被映射).

应用程序被垃圾收集.

我有时间深入研究这个问题，以下是我的发现:

1. 该问题仅发生在垃圾收集过程中.
2. 当从 NSMutableData 分配失败时，普通的 malloc 仍然成功分配相同数量的内存.
3. 当整体物理内存接近零(即将发生交换)时，错误总是发生.

我假设 NSData 在垃圾收集下运行时使用 NSAllocateCollectable 来执行分配而不是 malloc.

我的结论是，当物理内存不足时，收集器无法分配大块内存.再说一遍，我不明白.

解决方案

答案在于libauto.

从 OS X 10.6 开始，为 64 位平台上的垃圾收集内存分配了 8 Gb 的空间.对于大型分配 (>=128k) 和小型 (<2048b) 或中等 (<128k) 分配，该领域被减半.

因此，在 10.6 上，您有 4Gb 的内存可用于大量垃圾收集内存的分配.在 10.5 上，arena 的大小为 32Gb，但是Apple 在 10.6 上将该大小降低到 8Gb.>

I'm building a photo book layout application. The application frequently decompresses JPEG images into in-memory bitmap buffers. The size of the images is constrained to 100 megapixels (while they usually do not exceed 15 megapixels).

Sometimes memory allocations for these buffers fail: [[NSMutableData alloc] initWithLength:] returns nil. This seems to happen in situations where the systems's free physical memory approaches zero.

My understanding of the virtual memory system in Mac OS X was that an allocation in a 64 bit process virtually (sic) can't fail. There are 16 exabyte of address space of which I'm trying to allocate a maximum of 400 megabytes at a time. Theoretically I could allocate 40 billion of these buffers without hitting the hard limit of the available address space. Of course practical limits would prevent this scenario as swap space is constrained by the boot volume's size. In reality I'm only making very few of these allocations (less than ten).

What I do not understand is the fact that an allocation fails, no matter how low physical memory is at that point. I thought that—as long as there's swap space left—memory allocation would not fail (as the pages are not even mapped at this point).

The application is garbage collected.

Edit:

I had time to dig into this problem a little further and here are my findings:

1. The problem only occurs in a garbage collected process.
2. When the allocation from NSMutableData fails, a plain malloc still succeeds to allocate the same amount of memory.
3. The error always happens when overall physical memory approaches zero (swapping is about to take place).

I assume NSData uses NSAllocateCollectable to perform the allocation instead of malloc when running under garbage collection.

My conclusion from all that is that the collector is unable to allocate big chunks of memory when physical memory is low. Which again, I don't understand.

解决方案

The answer lies in the implementation of libauto.

As of OS X 10.6 an arena of 8 Gb is allocated for garbage collected memory on 64-bit platforms. This arena is cut in half for large allocations (>=128k) and small (<2048b) or medium (<128k) allocations.

So in effect on 10.6 you have 4Gb of memory available for large allocations of garbage collected memory. On 10.5 the arena had a size of 32Gb, but Apple lowered that size to 8Gb on 10.6.

这篇关于在 Mac OS X 上的 64 位进程中，中等大小的内存分配如何失败?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！