Python多进程分析 [英] Python multiprocess profiling

问题描述

我正在努力弄清楚如何分析一个简单的多进程python脚本

I'm struggling to figure out how to profile a simple multiprocess python script

import multiprocessing
import cProfile
import time
def worker(num):
    time.sleep(3)
    print 'Worker:', num

if __name__ == '__main__':
    for i in range(5):
        p = multiprocessing.Process(target=worker, args=(i,))
        cProfile.run('p.start()', 'prof%d.prof' %i)

我正在启动5个进程，因此cProfile生成5个不同的文件.在每个方法的内部，我希望看到我的方法'worker'大约需要3秒钟才能运行，但我只看到了'start'方法中正在发生的事情.

I'm starting 5 processes and therefore cProfile generates 5 different files. Inside of each I want to see that my method 'worker' takes approximately 3 seconds to run but instead I'm seeing only what's going on inside the 'start'method.

如果有人可以向我解释这一点，我将不胜感激.

I would greatly appreciate if somebody could explain this to me.

import multiprocessing
import cProfile
import time
def test(num):
    time.sleep(3)
    print 'Worker:', num

def worker(num):
    cProfile.runctx('test(num)', globals(), locals(), 'prof%d.prof' %num)


if __name__ == '__main__':
    for i in range(5):
        p = multiprocessing.Process(target=worker, args=(i,))
        p.start()

推荐答案

您正在对流程启动进行性能分析，这就是为什么您只看到您说的p.start()中发生了什么的原因，而p.start()返回后，子流程开始.您需要在worker方法内部进行配置，该方法将在子流程中调用.

You're profiling the process startup, which is why you're only seeing what happens in p.start() as you say—and p.start() returns once the subprocess is kicked off. You need to profile inside the worker method, which will get called in the subprocesses.

这篇关于Python多进程分析的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！