堆栈内存未释放 [英] Stack memory not released

问题描述

我有以下循环，它从此处的实现中弹出我拥有的 C++ 并发队列.https://juanchopanzacpp.wordpress.com/2013/02/26/并发队列-c11/

I have the following loop, which pops a C++ concurrent queue I have, from the implementation here. https://juanchopanzacpp.wordpress.com/2013/02/26/concurrent-queue-c11/

while (!interrupted)
{
    pxData data = queue->pop(); 
    if (data.value == -1)
    { 
        break; // exit loop on terminating condition
     }
    usleep(7000); // stub to simulate processing
}

我正在使用 CentOS7 中的系统监视器查看内存历史记录.从队列中读取值后，我试图释放队列占用的内存.但是，随着以下 while 循环的运行，我没有看到内存使用量下降.我已经验证队列长度确实下降了.

I am looking at the memory history using System Monitor in CentOS7. I'm trying to free up the memory taken up by the queue, after reading the value from the queue. However, as the following while loop runs, I don't see the memory usage going down. I've verified that the queue length does go down.

但是，当遇到 -1 并且循环退出时，它确实会下降.(程序仍在运行)但是我不能有这个，因为usleep在哪里，我想做一些密集处理.

It does go down, however, when -1 is encountered and the loop exits. (program is still running) But I can't have this, because where usleep is, I want to do some intensive processing.

问题:为什么数据占用的内存没有被释放?(根据系统监视器)当变量超出范围时，堆栈分配的内存不应该被释放吗?

Question: Why doesn't the memory occupied by data get free-ed? (according to System Monitor) Isn't the stack allocated memory supposed to be free-ed when the variable goes out of scope?

struct 定义如下，并在程序开始时填充.

The struct is defined as follows, and populated at the beginning of the program.

typedef struct pxData
{
  float value; // -1 value terminates the loop
  float x, y, z;
  std::complex<float> valueData[65536];
} pxData;

它填充了 ~10000 pxData，大致相当于 5GB.系统只有~8GB.因此，释放内存用于在系统中进行其他处理非常重要.

It's populated with ~10000 pxData, which roughly translates to 5GB. System only has ~8GB. So it's important that the memory is free-ed up for doing other processing in the system.

推荐答案

这里有一些事情在起作用.

There are a few things at play here.

首先，您需要了解，仅仅因为您的程序使用"了 5 GB 的内存并不意味着只有 3 GB 的 RAM 留给其他程序使用.虚拟内存意味着这 5 GB 可能只有 1 GB 的实际常驻"数据，另外 4 GB 可能实际上在磁盘上而不是在 RAM 中.因此，在查看程序时，重要的是查看常驻集大小"而不是虚拟大小".并请注意，如果您的系统实际运行的 RAM 不足，操作系统可能会通过调出"某些程序的内存来缩小某些程序的 RSS.因此，不要太担心系统监视器中出现5 GB"——如果您遇到真正的、具体的性能问题，请放心.

First, you need to understand that just because your program is "using" 5 GB of memory does not mean that there are only 3 GB of RAM left for other programs. Virtual memory means that those 5 GB might be only 1 GB of actual "resident" data, and the other 4 GB may actually be on disk rather than in RAM. So it's important to look at the "resident set size" rather than the "virtual size" when you're looking at your program. And note that if your system actually runs low on RAM, the OS may shrink the RSS of some programs by "paging out" some of their memory. So don't worry too much about "5 GB" appearing in the system monitor--worry if you have a real, concrete performance problem.

第二个方面是为什么当您从队列中删除项目时，您的虚拟大小不会减少.我们可以猜测，您通过使用 malloc 或 new 一一创建它们，然后将它们推到队列的后面来将这些元素放入队列.这意味着您分配的第一个元素将首先从队列中出来.这反过来意味着当您排空 90% 的队列时，您的内存分配可能如下所示:

The second aspect is why your virtual size does not decrease as you remove items from the queue. We can guess that you put those elements into the queue by creating them with malloc or new one-by-one, then pushing them onto the back of the queue. This means that the first element you allocated will come out of the queue first. And that in turn means that when you have drained 90% of the queue, your memory allocation might look like this:

[program|------------------unused-------------------|pxData]

这里的问题是，在现实世界中，仅仅因为您free 或delete 某些东西并不意味着操作系统会立即回收该内存.事实上，它可能无法回收任何未使用的跨度，除非它们处于末尾"(即最近分配的).由于 C++ 没有垃圾收集功能，并且未经您的同意不能在内存中移动项目，因此您最终会在程序的虚拟内存中出现这个大洞".这个洞将用于满足未来的内存分配请求，但如果你没有，它只是坐在那里，直到队列完全空:

The problem here is that in the real world, just because you free or delete something does not mean the operating system instantly reclaims that memory. In fact, it may not be able to reclaim any unused spans unless they are at the "end" (i.e. most recently allocated). Since C++ does not have garbage collection and cannot move items around in memory without your consent, you end up with this big "hole" in your program's virtual memory. That hole would be used to satisfy future memory allocation requests, but if you haven't got any, it just sits there, until the queue is completely empty:

[program|------------------unused--------------------------]

然后系统能够将您的虚拟地址空间缩小:

Then the system is able to shrink your virtual address space back down:

[program]

这会让您回到起点.

如果你想解决"这个问题，一个选择是反向"分配你的内存，即将分配的最后一项放在队列的前面.

If you want to "fix" this, one option is to allocate your memory in "reverse", i.e. put the last items allocated into the front of the queue.

另一种选择是通过 mmap 为队列分配元素，例如对于大"的分配，Linux 会自动执行.您可以通过调用 mallopt(3)<来更改此阈值/code> 与 M_MMAP_THRESHOLD 并将其设置为比您的结构大小小一点.这使得分配彼此独立，因此操作系统可以单独回收它们.这种技术甚至可以应用于现有程序而无需重新编译，因此如果您需要在无法修改的程序中解决此问题，通常会很有用.

Another option is to allocate the elements for the queue via mmap, which is something that e.g. Linux will do automatically for allocations which are "large." You can change the threshold for this by calling mallopt(3) with M_MMAP_THRESHOLD and setting it to be a little bit smaller than your struct size. This makes the allocations independent of each other, so the OS can reclaim them individually. This technique can even be applied to existing programs without recompilation, so is often useful if you need to solve this problem in a program you cannot modify.

这篇关于堆栈内存未释放的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！