Python线程分析 [英] Profiling of Python threads

问题描述

我试图弄清楚如何在我的应用程序中测量几个python线程的性能.我目前有几项任务是根据用户输入在不同的线程上执行的，我想测量执行时间，甚至可能是每个线程的内存消耗.我尝试使用cProfile(在线程的每个实例上，然后将数据写入文件，然后聚合所有结果)，但效果有限.另外，我还有一个问题，就是有些阻塞的IO会扭曲我的结果.无论如何，是否可以有效地分析我的应用程序?

I am trying to figure out how to measure the performance of several python threads in my application. I currently have several tasks that are executing on different threads based on user input and I would like to measure the execution time, maybe even memory consumption of each of the threads. I have tried to use cProfile (on each instantiation of the thread, then I would write the data to a file and then aggregate all results) with limited success. Also, I have an added problem of having some blocking IO which is skewing my results. Is there anyway to effectively profile my application?

推荐答案

有几种不同的方法可以解决此问题. cProfile很不错，它是Python附带的，但是许多人认为多线程分析是一个问题.解决此问题的一种方法是为每个线程运行单独的cProfile实例，然后使用Stats.add组合结果.

There are a few different ways to tackle this problem. cProfile is great and comes with Python, however many people see multi-threaded profiling as an issue. One way of getting around this is by running separate instances of cProfile for each thread and then combining the results using Stats.add.

如果不是您希望的那样有用，另一种选择是使用 Yappi ，我在一些特殊的多线程情况下使用成功.它具有出色的文档，因此您设置它不会有太多麻烦.

Should that not be as useful as you'd hoped, another alternative could be to use Yappi, which I've had success using for a few special multi-threaded cases. It's got great documentation so you shouldn't have too much trouble setting it up.

有关特定于内存的性能分析，请查看 Heapy .但请注意，如果代码不好，它可能会创建一些您从未见过的最大的日志文件！

For memory specific profiling, check out Heapy. But be warned, it may create some of the largest log files you've ever seen if your code is bad!

这篇关于Python线程分析的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！