Python逐行内存分析器? [英] Python line-by-line memory profiler?
问题描述
我希望从大型Python代码库中生成函数运行过程中堆使用情况或内存分配的摘要.
I'm looking to generate, from a large Python codebase, a summary of heap usage or memory allocations over the course of a function's run.
我对 heapy 熟悉,并且为拍摄快照"很好在代码中的特定点进行堆放,但是我发现很难用它生成随时间变化的存储器"摘要.我还玩过 line_profiler ,但是它只适用于运行时,而不适用于内存.
I'm familiar with heapy, and it's served me well for taking "snapshots" of the heap at particular points in my code, but I've found it difficult to generate a "memory-over-time" summary with it. I've also played with line_profiler, but that works with run time, not memory.
我现在的后备选择是带有 massif 的Valgrind,但是缺少很多Heapy和line_profiler给出的上下文Python信息.后两者是否有某种组合,可以在Python程序的执行范围内提供内存使用或堆增长的感觉?
My fallback right now is Valgrind with massif, but that lacks a lot of the contextual Python information that both Heapy and line_profiler give. Is there some sort of combination of the latter two that give a sense of memory usage or heap growth over the execution span of a Python program?
推荐答案
我会使用 sys.settrace
在程序启动时注册自定义跟踪器功能.对于每一行代码,都会调用custom_trace_function.然后,您可以使用该功能将堆积的或 meliae 收集的信息存储在文件中,以供以后处理.
I would use sys.settrace
at program startup to register a custom tracer function. The custom_trace_function will be called for each line of code. Then you can use that function to store information gathered by heapy or meliae in a file for later processing.
这是一个非常简单的示例,该示例每秒将hpy.heap()的输出记录到纯文本文件中:
Here is a very simple example which logs the output of hpy.heap() each second to a plain text file:
import sys
import time
import atexit
from guppy import hpy
_last_log_time = time.time()
_logfile = open('logfile.txt', 'w')
def heapy_profile(frame, event, arg):
currtime = time.time()
if currtime - _last_log_time < 1:
return
_last_log_time = currtime
code = frame.f_code
filename = code.co_filename
lineno = code.co_firstlineno
idset = hpy().heap()
logfile.write('%s %s:%s\n%s\n\n' % (currtime, filename, lineno, idset))
logfile.flush()
atexit.register(_logfile.close)
sys.settrace(heapy_profile)
这篇关于Python逐行内存分析器?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!