Python Google App Engine无法释放对象数组的内存 [英] Python Google App Engine can not release memory of object array

问题描述

我使用GAE(谷歌应用程序引擎)，制作一个非常简单的Web应用程序. 在处理请求的方法中，我只是创建了一大堆对象. 在那之后，我删除了对该数组的所有引用. 之后，我叫gc.collect.

I use GAE(google app engine), make a very simple web application. in the method of processing request, i just create a big array of objects. After that, i delete all references to the array. After that, i call gc.collect.

但是当我长时间测试(发送请求)时，仪表板的内存使用量继续增加.

But when i test (send request) for a long time, the memory usage of Dashboard continue increase.

我看起来像是内存泄漏. 但是我认为代码还可以.

I look like memory leak. But i think the code is ok.

下面是示例代码.

from flask import Flask, request

import gc

app = Flask(__name__)

@app.route('/', methods=['POST'])
def hello():

    gc.enable()

    bigArr = []
    for x in range(10000):
        raw_data = request.get_data(cache=False)
        bigArr.append(raw_data)
        del raw_data

    print('len(bigArr):' + str(len(bigArr)))
    del bigArr
    gc.collect()

    return 'Hello World'


if __name__ == '__main__':
    app.run(host='0.0.0.0', port=80, debug=True)

App引擎配置: 运行时:python37 automatic_scaling: max_instances:1

App engine config: runtime: python37 automatic_scaling: max_instances: 1

这是内存使用情况的图像:

Here's the image of memory usage:

推荐答案

该图看起来好像您的内存使用情况继续增加".相反，它看起来很平整.如果您发生了显着的内存泄漏，则图形会上升.

That graph doesn't look as if your memory usage "continues to increase". Rather, it looks pretty flat. If you had a significant memory leak, the graph would go up instead.

Python进程需要从操作系统获取内存，然后使用该内存来存储Python对象.在对Python对象进行垃圾回收时，这些对象所占用的内存将对 Python进程免费，因为可以将新对象存储在该处.但是对于操作系统而言，该内存仍归Python进程所有，因此正在使用中.我想您的图形显示了该Python进程的内存使用情况.

The Python process needs to get memory from the operating system, and then uses that memory to store your Python objects. When Python objects are garbage collected, the memory occupied by those objects becomes free to the Python process, because new objects can be stored there. But to the operating system, that memory is still owned by the Python process, so it is in use. I suppose your graph shows the memory usage of that Python process.

从操作系统获得的内存的请求块要比单个Python对象所需的块大得多.而且它也必须以更大的块返回.当分配Python对象并随后对其进行垃圾回收时，剩余的活动对象将散布在巨大的内存块中.如果Python进程想要将内存释放回操作系统，则必须将所有对象移到一个紧凑的区域，这样，一个巨大的连续区域就可以释放了.仅需保留内存并根据需要重新使用它，就可以使该过程更容易，更快捷.

The memory obtained from the operating system is requested in much larger chunks than needed for a single Python object. And it would have to be returned in lager chunks as well. When Python objects are allocated and later garbage-collected, the remaining live objects are spread across the huge memory chunk. If the Python process wanted to release memory back to the operating system, it would have to move all the objects to a compact area, so that a huge, contiguous area becomes free. It's easier for the process, and faster, to just hold on to the memory and re-use it as needed.

这篇关于Python Google App Engine无法释放对象数组的内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！