C进程中的Lua内存泄漏 [英] Lua memory leak in C process

问题描述

我有一个与Lua一起运行的C程序.

I have a C program running with Lua.

尽管我尝试使用lua_gc()获取并控制Lua的内存使用量，但C进程的内存使用量仍然很高.即使Lua说它仅使用4MB内存，C进程也会使用150MB以上的内存.

Although I try to use lua_gc() to get and control memory usage of Lua, C process memory usage is still high. The C process uses over 150MB memory even though Lua say that it uses only 4MB memory.

我也尝试使用l_alloc()函数跟踪Lua的内存分配，但是结果与通过调用lua_gc(LUA_GCCOUNT)和lua_gc(LUA_GCCOUNTB)告知Lua的内存使用情况相同.

I also try to use my l_alloc() function to track Lua memory allocation, but the result is the same with the memory usage Lua told by calling lua_gc(LUA_GCCOUNT) and lua_gc(LUA_GCCOUNTB).

在调用lua_close()关闭Lua环境后，进程内存已关闭并且看起来不错. 因此，我认为丢失的内存"仍然是由Lua而不是C程序控制的.

After calling lua_close() to close the Lua environment, the process memory is down and looks fine. Therefore, I think the 'lost memory' is still controlled by Lua not C program.

这是示例C代码. 它创建一个Lua环境，调用Lua函数清除数据，然后检查内存使用情况.

Here is the sample C code. It creates a Lua environment, calls Lua function to purge data and then check memory usage.

int main()
{
    int rc;
    uint64_t gc_mem_usage;

    lua_State* Lua = luaL_newstate();
    luaL_openlibs(Lua);

    lua_gc(Lua, LUA_GCSTOP, 0);
    luaL_dofile(Lua, "test.lua");

    gc_mem_usage = ((uint64_t)lua_gc(Lua, LUA_GCCOUNT, 0) << 10) + lua_gc(Lua, LUA_GCCOUNTB, 0);
    printf("Lua mem usage: [%" PRIu64 "] Bytes\n", gc_mem_usage);

    lua_getglobal(Lua, "command_handler");
    lua_pushstring(Lua, "CC");
    rc = lua_pcall(Lua, 1, 0, 0);
    if (rc != 0 ) {
        printf("function error\n");
        return;
    }

    lua_settop(Lua, 0);

    // do full gc
    lua_gc(Lua, LUA_GCCOLLECT, 0);
    lua_gc(Lua, LUA_GCCOLLECT, 0); // I don't know why it has different result by calling full gc twice
    sleep(1);

    printf("-------------After GC ----------------------\n");
    gc_mem_usage = ((uint64_t)lua_gc(Lua, LUA_GCCOUNT, 0) << 10) + lua_gc(Lua, LUA_GCCOUNTB, 0);
    printf("Lua mem usage: [%" PRIu64 "] Bytes\n", gc_mem_usage);

    // infinite-loop
    while(1);
}

Lua示例代码:

local abc = {}

function command_handler(cmd)
    if (cmd == "CC") then
        abc = {}
    end
end

for i =0, 2000000 do
    abc[i] = "ABC" .. i .. "DEF"
end

输出:

Lua mem usage: [204913817] Bytes
-------------After GC ----------------------
Lua mem usage: [4219342] Bytes

输出告诉我，GC之后Lua的内存使用率下降了，但是通过连续检查top，此C进程的内存使用率仍然很高(193.7MB).

The output told me that Lua memory usage is down after GC, but the memory usage of this C process is still very high (193.7MB) by checking atop continuously.

 PID MINFLT MAJFLT      VSTEXT  VSIZE  RSIZE  VGROW  RGROW  MEM CMD     1/1
4622      1      0          3K 193.7M 183.9M     0K     4K  18% a.out

有什么解决方案可以减少C进程的内存使用?

Is there any solution to reduce the C process memory usage?

我的环境是在Ubuntu/CentOS中运行的Lua 5.1.4.

My environment is Lua 5.1.4 running in Ubuntu/CentOS.

推荐答案

Lua通过调用提供的释放函数(默认为realloc(block, 0))忠实地释放无法访问的对象.看起来libc分配器正在努力返回未使用的内存，这可能是由于碎片过多所致.查看strace输出(在64位Debian 6上，我与Lua 5.1.4的数字大致相同)，C运行时选择使用brk进行分配，并且增量较小，但不取消分配(调用brk较低的值).但是，如果在进入无限循环之前插入malloc_trim(M_TOP_PAD)，您会在top输出中看到驻留大小急剧下降到〜5M，并且strace揭示确实使用brk修剪了数据段.在这种情况下，使用自定义分配器(例如基于池的分配器)或调整malloc参数可能会有所帮助.

Lua faithfully frees unreachable objects by calling the supplied deallocation function (by default realloc(block, 0)). It looks like libc allocator is struggling to return unused memory, possibly due to high fragmentation. Looking at the strace output (I've got roughly the same numbers with Lua 5.1.4 on 64-bit Debian 6), C runtime chooses to allocate using brk with small increments, but no deallocation (calling brk with lower value) follows. However, if you insert malloc_trim(M_TOP_PAD) before entering infinite loop, you'll see in the top output that resident size drops drastically to ~5M and strace reveals that data segment was indeed trimmed with brk. Using custom allocator (e.g. pool-based) or tuning malloc parameters will probably help in this situation.

这篇关于C进程中的Lua内存泄漏的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！