在App Engine后端实例上完成请求后，什么时候可以释放内存? [英] When will memory get freed after completing the request on App Engine Backend Instances?
问题描述
场景-
我正在App Engine上运行B *实例.我有一个与后台ETL相关的任务(用python 编写)安排为App Engine上的cron作业. 时间到了，cron会发起一个http请求以启动任务，并在不返回响应的情况下运行直到任务完成为止. 执行任务时，通常会消耗"X" MB的RAM.任务完成并返回200 OK之后，App Engine实例监视仍显示正在使用的"X" MB RAM.
I am running B* instances on App Engine. I've a background ETL related task(written in python) scheduled as a cron job on App Engine. When time arrives, cron initiates a http request to start the task and runs without returning a response till the task gets completed. When task was executing, it was typically consuming "X" MB of RAM. After the task got finished and returned 200 OK, App Engine Instance monitoring is still showing "X" MB of RAM in use.
请帮助我理解以下内容-
Please help me understand the following -
- 如果一个实例仅运行一个任务并完成后，什么时候该任务消耗的内存将被释放?
- 是否需要运行
gc.collect()来显式调用垃圾回收器以释放RAM? - 释放RAM的唯一方法是重新启动实例?
- If an instance is running only one task and after completing it, when will memory get freed that was consumed by this task?
- Do I need to run
gc.collect()to call the garbage collector explicitly to free up RAM ? - The only way to free up RAM is to restart the instance ?
PS:这与NDB完全无关，我的任务是从Bigquery接收输入，执行一些ETL操作，然后将其流式传输到Bigquery.
推荐答案
根据我对一个应用程序的观察，该应用程序为StringIO操作使用了很多内存:
From my observations with an app using lots of memory for StringIO operations:
-
明确地调用
gc.collect()并没有明显的帮助(我什至怀疑我确实有内存泄漏，但事实并非如此)
explicitly calling
gc.collect()didn't noticeably help (I even suspected for a while that I actually have memory leaks, but it wasn't the case)
在每个请求之后都不会释放内存，但是，如果实例保持活动的时间足够长而又没有内存用尽，它最终会
似乎不时被释放.易于测试-只需增加两次请求之间的时间即可减少可用内存的消耗率.但是我无法找出一个可用的模式.请注意，只有在升级到B2实例后，我才观察到此情况，我的B1实例的内存不足速度过快，我从没注意到与它们相关的释放事件.
the memory is not freed after each and every request, but, if the instance remains alive long enough without running out of memory it does eventual
appears to be freed now and then. Easy to test - just increase the time between requests to reduce the free memory draining rate. But I couldn't figure out a usable pattern. Note that I observed this only after upgrading to B2 instances, my B1 instances were running out of memory too fast, I never noticed a freeing event with them.
使用具有更多内存的实例类我尝试作为一种变通办法来解决我的实例最终用完内存的问题)有所帮助-内存似乎被更频繁地释放了. 可能是因为这些实例还具有更快的CPU(但这只是猜测).
using an instance class with more memory (which I tried as a workaround for my instances eventually running out of memory) helped - the memory appeared to be freed more often. It might be because these instances also have a faster CPU (but that's just guesswork).
这篇关于在App Engine后端实例上完成请求后，什么时候可以释放内存?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
