适用于Python的CPU火焰图 [英] CPU Flame Graphs for Python

问题描述

Brendan Gregg的 CPU火焰图是一种可视化一段时间内CPU使用率的方法.时间基于调用堆栈.

Brendan Gregg's CPU Flame Graphs are a way of visualising CPU usage over a period of time based on call stacks.

他的 FlameGraph github项目提供了一种独立于语言的方式来绘制这些图:

His FlameGraph github project provides a language-independent way to plot these graphs:

对于每种语言，FlameGraph都需要一种以如下形式的行形式提供堆栈输入的方法:

For each language, FlameGraph requires a way of providing stack input in the form of lines like this:

grandparent_func;parent_func;func 42

这意味着在运行func函数时观察到了所插入的程序，该函数从parent_func调用，然后又从顶级函数grandparent_func调用.它说该调用堆栈被观察了42次.

This means that the instrumented program was observed running function func, where that was called from parent_func, in turn called from top-level function grandparent_func. It says that call stack was observed 42 times.

如何从Python程序中收集堆栈信息并将其提供给FlameGraph?

How can I gather stack information from Python programs and provide it to FlameGraph?

要点:要如何扩展才能显示C和Python堆栈，甚至可以显示到Linux上的内核(类似于布伦丹网站上的某些Java和node.js火焰图) )?

For bonus points: How can that be extended so that both the C and Python stack is shown, or even down to the kernel on Linux (in a similar way to some of the Java and node.js flame graphs on Brendan's website)?

推荐答案

也许您可以尝试 sys.setprofile ，这是标准python分析器 profile 和 cProfile .此方法为每个函数(包括C-API的那些函数)的调用"和返回"事件设置一个钩子.

Maybe you can try sys.setprofile, which is the core for the standard python profiler profile and cProfile. This method sets a hook to the "call" and "return" events of every function, including those functions of C-API.

系统的配置文件函数的调用与系统的跟踪函数类似(请参见settrace())，但是不会为每个已执行的代码行调用(仅在调用和返回时调用)，即使在设置了例外).

The system’s profile function is called similarly to the system’s trace function (see settrace()), but it isn’t called for each executed line of code (only on call and return, but the return event is reported even when an exception has been set).

下面是一个有效的示例:

Below is a working example:

from time import clock 
t0 = clock()

def getFun(frame):
    code = frame.f_code 
    return  code.co_name+' in '+code.co_filename+':'+str(code.co_firstlineno)

def trace_dispatch(frame, event, arg):
    if event in [ "c_call" , 'call', 'return', 'c_return']:
        t = int((clock()-t0)*1000)
        f = frame
        stack=[]
        while(f):
          stack.insert( 0,getFun(f) )
          f = f.f_back
        print event, '\t', '; '.join(stack), '; ', t

import sys
sys.setprofile(trace_dispatch)
try:
    execfile('test.py')
finally:
    sys.setprofile(None)

Test.py

def f(x):
    return x+1
def main(x):
    return f(x)
main(10)

这将打印出来

c_call    0
call      <module> in test.py:2 ;  1
call      <module> in test.py:2; main in test.py:5 ;  1
call      <module> in test.py:2; main in test.py:5; f in test.py:2 ;  5
return    <module> in test.py:2; main in test.py:5; f in test.py:2 ;  8
return    <module> in test.py:2; main in test.py:5 ;  11
return    <module> in test.py:2 ;  14
c_return  18
c_call    21

在此处中查看更全面的分析功能.

See a more comprehensive profiling function here.

您无法在python解释器中访问C堆栈.必须使用支持C/C ++的调试器或探查器.我会建议 gdb python .

You cannot access the C stack within the python interpreter. It is necessary to use a debugger or profiler that supports C/C++. I would recommand gdb python.

这篇关于适用于Python的CPU火焰图的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！