如何解决内存碎片 [英] How to solve Memory Fragmentation

问题描述

我们偶尔遇到问题，由于内存分配失败，我们长期运行的服务器进程（在Windows Server 2003上运行）抛出异常。我们怀疑这些分配由于内存碎片而失败。

We've occasionally been getting problems whereby our long-running server processes (running on Windows Server 2003) have thrown an exception due to a memory allocation failure. Our suspicion is these allocations are failing due to memory fragmentation.

因此，我们一直在寻找一些替代的内存分配机制，可以帮助我们，我希望有人可以告诉我最好的一个：

Therefore, we've been looking at some alternative memory allocation mechanisms that may help us and I'm hoping someone can tell me the best one:

1）使用Windows 低碎片堆

1) Use Windows Low-fragmentation Heap

2）jemalloc - 用于 Firefox 3

2) jemalloc - as used in Firefox 3

3）Doug Lea的 malloc

3) Doug Lea's malloc

我们的服务器进程是使用跨平台C ++代码开发的，所以任何解决方案都将理想地跨平台（do * nix操作系统遭受这种类型的内存碎片？ ）。

Our server process is developed using cross-platform C++ code, so any solution would be ideally cross-platform also (do *nix operating systems suffer from this type of memory fragmentation?).

此外，我认为LFH现在是Windows Server 2008 / Vista的默认内存分配机制吗？...我的当前问题消失if我们的客户只需升级他们的服务器os？

Also, am I right in thinking that LFH is now the default memory allocation mechanism for Windows Server 2008 / Vista?... Will my current problems "go away" if our customers simply upgrade their server os?

推荐答案

首先，我同意其他海报建议资源泄露。你真的想先排除。

First, I agree with the other posters who suggested a resource leak. You really want to rule that out first.

希望你目前使用的堆管理器有一种方法来转储堆中可用的实际总可用空间所有自由块）以及它所划分的块的总数。如果平均自由块大小与堆中的总可用空间相比相对较小，那么您确实有分段问题。或者，如果你可以转储最大的空闲块的大小，并将其与总可用空间进行比较，那将完成同样的事情。如果您正在进行碎片化，则最大的可用区块相对于所有区块中可用的所有可用 空间而言是小的。

Hopefully, the heap manager you are currently using has a way to dump out the actual total free space available in the heap (across all free blocks) and also the total number of blocks that it is divided over. If the average free block size is relatively small compared to the total free space in the heap, then you do have a fragmentation problem. Alternatively, if you can dump the size of the largest free block and compare that to the total free space, that will accomplish the same thing. The largest free block would be small relative to the total free space available across all blocks if you are running into fragmentation.

要非常清楚上述内容，在所有情况下，我们都在谈论堆中的 自由块，而不是堆中分配的块。在任何情况下，如果不满足上述条件，那么你有某种泄漏的情况。

To be very clear about the above, in all cases we are talking about free blocks in the heap, not the allocated blocks in the heap. In any case, if the above conditions are not met, then you do have a leak situation of some sort.

因此，一旦你有排除了泄漏，你可以考虑使用更好的分配器。问题中建议的 Doug Lea的malloc 是一般用途应用程序的非常好的分配器，并且非常强大。换句话说，它已经经过时间测试，适用于大多数任何应用程序。然而，没有算法对于所有应用程序是理想的，并且任何管理算法方法都可以通过正确的病理条件来破坏它的设计。

So, once you have ruled out a leak, you could consider using a better allocator. Doug Lea's malloc suggested in the question is a very good allocator for general use applications and very robust most of the time. Put another way, it has been time tested to work very well for most any application. However, no algorithm is ideal for all applications and any management algorithm approach can be broken by the right pathelogical conditions against it's design.

为什么会出现碎片问题？ - 由应用程序的行为导致的碎片问题的来源是 ，并且在相同的内存领域有很大不同的分配生命周期。也就是说，一些对象被定期分配和释放，而其他类型的对象在同一个堆中持续延长的时间段。想到更长寿命的对象，因为在竞技场的较大区域中打洞，从而防止

Why are you having a fragmentation problem? - Sources of fragmentation problems are caused by the behavior of an application and have to do with greatly different allocation lifetimes in the same memory arena. That is, some objects are allocated and freed regularly while other types of objects persist for extended periods of time all in the same heap.....think of the longer lifetime ones as poking holes into larger areas of the arena and thereby preventing the coalesce of adjacent blocks that have been freed.

为了解决这种类型的问题，你可以做的最好的事情是逻辑上将堆分成子领域，其中生命周期更长类似。实际上，你想要一个临时堆和一个持久堆或堆对类似生命周期的东西。

To address this type of problem, the best thing you can do is logically divide the heap into sub arenas where the lifetimes are more similar. In effect, you want a transient heap and a persistent heap or heaps that group things of similar lifetimes.

一些人建议另一种方法来解决这个问题，以使分配大小更相似或相同，但这不太理想，因为它创建了一种不同类型的碎片称为内部碎片 - 这实际上是浪费的空间，你通过在块中分配更多的内存，比你需要。

Some others have suggested another approach to solve the problem which is to attempt to make the allocation sizes more similar or identical, but this is less ideal because it creates a different type of fragmentation called internal fragmentation - which is in effect the wasted space you have by allocating more memory in the block than you need.

此外，使用良好的堆分配器，如Doug Lea，使块大小更相似是不必要的，因为分配器已经做了两个大小分块方案的权力将使得完全不必人为地调整传递给malloc（）的分配大小 - 实际上，他的堆管理器为您自动执行的程序比应用程序能够进行调整要强得多。

Additionally, with a good heap allocator, like Doug Lea's, making the block sizes more similar is unnecessary because the allocator will already be doing a power of two size bucketing scheme that will make it completely unnecessary to artificially adjust the allocation sizes passed to malloc() - in effect, his heap manager does that for you automatically much more robustly than the application will be able to make adjustments.

这篇关于如何解决内存碎片的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！