如何迭代所有malloc块（glibc） [英] How to iterate all malloc chunks (glibc)

问题描述

我试图遍历所有领域中的所有malloc_chunk。 （基于核心文件进行调试，针对内存泄漏和内存损坏调查）

据我所知，每个竞技场都有 top_chunk 指向顶部块在基于 top_chunk 的一个竞技场内部，基于代码（glibc / malloc / malloc），存在 prev_size 和 size 。 c）：

我可以得到前面的连续块，然后在一个舞台上循环所有的块。 （我可以统计块的大小和数量，如WinDBG：！heap -stat -h），也可以基于 prev_size 和 size ，我可以检查chunk已损坏或不存在。

在竞技场（malloc_state）中，有一个成员变量： next 指向下一个竞技场。然后我可以循环所有的舞台的块。

但是我遇到了一个问题，如果块未分配，prev_size无效，如何获得以前的malloc_chunk？或者这种方式是不正确的。

---

问题背景：

我们拥有的内存泄漏错误是在几个在线数据节点（我们的项目是分布式存储集群）中报告的内存泄漏。

我们做了什么，结果如何：

1. 我们使用valrgind来重现测试集群中的错误，但不幸的是我们什么也得不到。

我试图调查更多关于堆的信息，试图分析堆块并遵循之前在WinDBG中的做法（它有非常有趣的堆命令挖掘内存泄漏和内存损坏），但是我被问到的问题挡住了。 我们使用valgrind-massif来分析分配我认为这是非常详细和有趣的，可以显示哪些分配需要多少内存）。 Massif显示了几条线索，我们按照这个步骤检查代码，最终发现泄漏（地图非常巨大，并且正确使用它，但是我会在持有者类的析构函数中清除，这就是为什么valgrind不会报告这种情况）。 / p>

我将深入了解gdb-heap源代码，以便了解更多关于glic malloc结构的信息。

解决方案

首先，在深入讨论 malloc 的实现细节之前，您的时间可能会更好花费在像 valgrind 这样的工具上，甚至可以在 MALLOC_CHECK _ 环境变量来让内部堆一致性检查为你工作。

但是，因为你问过....

glibc's malloc.c 有一些关于查看前一个块的有用评论。

一些特别有趣的是：

/ *请注意，我们甚至无法查看prev，除非它不是inuse * /

$ b $ 如果prev_inuse被设置为任何给定的块，那么你不能确定前一个块的大小，甚至可能在尝试时发生内存寻址错误。

这个只是malloc implimentation的一个限制。当前面的块正在使用时，存储大小的页脚会被分配的用户数据使用。

虽然它不能帮助你的情况，你可以通过追踪 prev_inuse 宏来检查前一个块是否正在使用中。

  #define PREV_INUSE 0x1 
 #define prev_inuse（p）（（p） - > size& PREV_INUSE）
  

它检查当前块大小的低位。 （所有块大小都可以被4整除，所以低2位可以用于状态。）这将帮助你在进入无人区之前停止迭代。

不幸的是，在访问每个块之前，你仍然会尽早终止你的循环。

如果你真的想迭代所有的块，我建议你从 malloc_state :: top 开始，然后按照 next_chunk 直到 next_chunk 指向顶部。

I'm trying to iterate all the malloc_chunk in all arenas. (debugging based on core file, for memory leak and memory corruption investigation)

As i know each arena have top_chunk which point to the top chunk inside of one arena, based on top_chunk, inside of it, there's prev_size and size, based on the code (glibc/malloc/malloc.c): I can get the previous continuous chunks, and then loop all the chunks in one arena. (i can statistic the chunks with the size and the number, which like WinDBG: !heap -stat -h) and also based on prev_size and size, i can check the chunk is corrupt or not.

In arena(malloc_state), there's a member variable: next which point to next arena. Then i can loop all the arena's chunks.

But i met a problem is if the chunk is not allocated, the prev_size is invalid, how to get the previous malloc_chunk?? Or this way is not correct.

---

Question Background:

The memory leak bug we have is memory leak reported in several online data node(our project is distributed storage cluster).

What we did and result:

1. We use valrgind to reproduce the bug in test cluster, but unfortunately we get nothing.

2. I tried to investigate more about the heap, tried to analyze the heap chunk and follow the way which i did before in WinDBG(which have very interesting heap commands to digger the memory leak and memory corruption), but i was blocked by the Question which i asked.

3. We use valgrind-massif to analyze the allocation(which i think it's very detail and interesting, could show which allocation takes how much memory). Massif show several clues, we follow this and check code, finally found the leak(a map is very huge, and in-proper usage of it, but i would erase in holder-class's destructor, that's why valgrind not report this).

I'll digger more about the gdb-heap source code to know more about glic malloc structure.

解决方案

First, before digging into the implementation details of malloc, your time may be better spent with a tool like valgrind or even run under the MALLOC_CHECK_ environment variable to let the internal heap consistency checking do the work for you.

But, since you asked....

glibc's malloc.c has some helpful comments about looking at the previous chunk.

Some particularly interesting ones are:

/* Note that we cannot even look at prev unless it is not inuse */

And:

If prev_inuse is set for any given chunk, then you CANNOT determine the size of the previous chunk, and might even get a memory addressing fault when trying to do so.

This is just a limitation of the malloc implimentation. When a previous chunk is in use, the footer that would store the size is used by the user-data of the allocation instead.

While it doesn't help your case, you can check whether a previous chunk is in use by following what the prev_inuse macro does.

#define PREV_INUSE 0x1
#define prev_inuse(p) ((p)->size & PREV_INUSE)

It checks the low-order bit of the current chunk's size. (All chunk sizes are divisible by 4 so the lower 2 bits can be used for status.) That would help you stop your iteration before going off into no-man's land.

Unfortunately, you'd still be terminating your loop early, before visiting every chunk.

If you really want to iterate over all chunks, I'd recommend that you start at malloc_state::top and follow the next_chunk until next_chunk points to the top.

这篇关于如何迭代所有malloc块（glibc）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！