Linux分配器不释放小块内存 [英] Linux Allocator Does Not Release Small Chunks of Memory

问题描述

Linux glibc分配器似乎表现得很奇怪。希望，有人可以阐明这一点。这是我有的源文件：

The Linux glibc allocator seems to be behaving weirdly. Hopefully, someone can shed some light on this. Here is the source file that I have:

first.cpp：

first.cpp:

#include <unistd.h>
#include <stdlib.h>
#include <list>
#include <vector>

int main() {

  std::list<char*> ptrs;
  for(size_t i = 0; i < 50000; ++i) {
    ptrs.push_back( new char[1024] );
  }
  for(size_t i = 0; i < 50000; ++i) {
    delete[] ptrs.back();
    ptrs.pop_back();
  }

  ptrs.clear();

  sleep(100);

  return 0;
}

second.cpp：

second.cpp:

#include <unistd.h>
#include <stdlib.h>
#include <list>

int main() {

  char** ptrs = new char*[50000];
  for(size_t i = 0; i < 50000; ++i) {
    ptrs[i] = new char[1024];
  }
  for(size_t i = 0; i < 50000; ++i) {
    delete[] ptrs[i];
  }
  delete[] ptrs;

  sleep(100);

  return 0;
}

我编译了两个：

$ g++ -o first first.cpp
$ g++ -o second second.cpp

我先运行，在它睡觉后，我看到常驻内存大小：

I run first, and after it's sleeping, I see the resident memory size:

当我编译first.cpp，运行它，我看看内存与ps ：

When I compile first.cpp, and run it, I look at memory with ps:

$ ./first&
$ ps aux | grep first
davidw    9393  1.3  0.3  64344 53016 pts/4    S    23:37   0:00 ./first


$ ./second&
$ ps aux | grep second
davidw    9404  1.0  0.0  12068  1024 pts/4    S    23:38   0:00 ./second

注意常驻内存大小。首先，常驻内存大小为53016k。在第二，它是1024k。第一个从未因为某种原因释放分配给内核。

Notice the resident memory size. In first, the resident memory size is 53016k. in second, it is 1024k. First never released the allocations back to the kernel for some reason or another.

为什么第一个程序不会释放内存给第二个程序？我的理解，第一个程序使用链表，链表可能分配一些节点在同一页上作为我们释放的数据。然而，这些节点应该被释放，因为我们弹出这些节点，然后清除链表。如果你通过valgrind运行这些程序，它会回来没有内存泄漏。可能发生的是内存获取碎片在first.cpp中不在second.cpp。但是，如果页面上的所有内存都被释放，那么该页面不会被释放回内核？内存被释放回内核需要什么？如何修改first.cpp（继续将char *放在列表中），以便将内存释放给内核。

Why does the first program not relinquish memory to the kernel, but the second program does? I understand that the first program uses a linked list and the linked list probably allocates some nodes on the same page as the data we're freeing. However, those nodes should be freed, as we're popping those nodes off, then clearing the linked list. If you run either of these programs through valgrind, it comes back with no memory leaks. What is probably happening is memory gets fragmented in first.cpp that doesn't in second.cpp. However, if all memory on a page is freed, how does that page not get relinquished back to the kernel? What does it take for memory to get relinquished back to the kernel? How can I modify first.cpp (continuing to put the char*'s in a list) so that the memory is relinquished to the kernel.

推荐答案

这种行为是有意的，有一个可调阈值，glibc用来决定是否实际向系统返回内存，或者是否缓存以供以后重用。在你的第一个程序中，你使用每个 push_back 进行大量的小分配，这些小的分配不是一个连续的块，并且可能低于阈值，所以不要返回到OS。

This behaviour is intentional, there is a tunable threshold that glibc uses to decide whether to actually return memory to the system or whether to cache it for later reuse. In your first program you make lots of small allocations with each push_back and those small allocations are not a contiguous block and are presumably below the threshold, so don't get returned to the OS.

调用 malloc_trim（0） 清除列表后应立即导致glibc
将空闲内存的最顶层区域返回到系统（需要 sbrk 系统调用下次需要内存
。）

Calling malloc_trim(0) after clearing the list should cause glibc to immediately return the top-most region of free memory to the system (requiring a sbrk system call next time memory is needed.)

如果你真的需要重写默认行为除非分析显示它实际上有帮助），那么你应该使用strace和/或实验
mallinfo 到
查看您的程序中实际发生的情况，也许可以使用 mallopt 至
调整将内存返回系统。

If you really need to override the default behaviour (which I wouldn't recommend unless profiling reveals it actually helps) then you should probably use strace and/or experiment with mallinfo to see what's actually happening in your program, and maybe using mallopt to adjust the threshold for returning memory to the system.

这篇关于Linux分配器不释放小块内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！