如何按请求记录Django应用程序的内存使用情况 [英] How to log memory usage of an Django app per request
问题描述
您知道一种有效的方法来记录每个请求django应用程序的内存使用情况吗?
Do you know about an efficient way to log memory usage of a django app per request ?
我有一个apache/mod_wsgi/django堆栈，该堆栈通常运行良好，但有时一个进程最终会消耗大量内存.服务器最终缺少内存，交换很多，并且服务大大降低了速度.
I have an apache/mod_wsgi/django stack, which runs usually well, but sometimes one process ends up eating a huge lot of memory. The servers ends up being short on mem, swapping a lot, and services are dramatically slowed down.
这种情况很难解决，因为我不知道应将此行为归咎于哪个请求，我无法重现该请求.
This situation is quite hard to fix because I don't know which request is to be blamed for this behavior, I can't reproduce it.
我想在生产环境中部署一些东西，以便在每次请求之前和之后记录该进程的内存使用情况，并且开销最小.
I'd like to have something deployed in production which logs the memory usage of the process before and after each request, with minimal overhead.
在我重新发明轮子之前，我的djangoists社区成员是否知道解决此问题的任何现有解决方案? 建议，中间件，代码段或apache日志配置值得赞赏.
Before I start reinventing the wheel, do the community of my fellow djangoists know any existing solution to address this problem ? Advices, middleware, snippet or maybe apache log configuration appreciated.
我不需要的是:
- 一组开发阶段的概要分析/调试工具，我已经知道了一些，如果我知道要概要分析/调试的东西，我会使用它们，它看起来有点太多，无法永久监视生产中运行的服务.最重要的是，这些工具通常显示的是碎片的mem使用情况报告，将其切成碎片，这对于确定错误的请求确实很有帮助.
- 关于如何优化django应用程序的内存使用的一般性建议，读起来总是很好，但是这里的想法是如何有效地跟踪需要优化的请求".
我最近的搜索结果:
- Django / WSGI - How to profile partial request? My profiling tools are per-request but app runs out of memory before then
- Django memory usage going up with every request
- Average php memory usage per request?
推荐答案
一个用于跟踪内存使用情况并立即生成可用结果的Django中间件，需要同时挂接流程请求和流程响应.换句话说，查看请求的开始和结束之间的差异，如果超过某个阈值，则记录警告.
A Django middleware for tracking memory usage and generating a usable result immediately, needs to hook both process request and process response. In other words, look at difference between start and finish of request and log a warning if exceeds some threshold.
一个完整的中间件示例是:
A complete middleware example is:
import os
import psutil
import sys
THRESHOLD = 2*1024*1024
class MemoryUsageMiddleware(object):
def process_request(self, request):
request._mem = psutil.Process(os.getpid()).memory_info()
def process_response(self, request, response):
mem = psutil.Process(os.getpid()).memory_info()
diff = mem.rss - request._mem.rss
if diff > THRESHOLD:
print >> sys.stderr, 'MEMORY USAGE %r' % ((diff, request.path),)
return response
这需要安装"psutil"模块来进行内存计算.
This requires the 'psutil' module to be installed for doing memory calculation.
是蛮力的，可能会在多线程系统中导致误报.由于是延迟加载，随着内容加载，您还会看到针对新流程的前几个请求会触发它.
Is brute force and can lead to false positives in a multithread system. Because of lazy loading, you will also see it trigger on first few requests against new process as stuff loads up.
这篇关于如何按请求记录Django应用程序的内存使用情况的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
