我们如何描述每个Celery任务的内存使用情况? [英] How can we profile memory usage of each Celery task?

问题描述

我使用pytest进行测试，而pytest又执行了一些celery任务.但是，当我使用

I run tests using pytest which in turn runs a couple of celery tasks. But, when I use

mprof run --include-children py.test test.py

我无法跟踪在芹菜工作者内部运行的每个芹菜任务消耗的内存.我能看到py.test进程消耗了50 MB的RAM(我可以在mprof图中看到)，而celery工人消耗了大约600-700 MB的RAM，这并没有反映在mprof图中.如何绘制由py.test触发的每个Celery任务的内存消耗?

I am not able to track the memory consumed by each celery task that is running inside a celery worker. All I could see 50 MB of RAM being consumed by the py.test process (which I am able see in the mprof plot) whereas the celery worker consumes around 600-700 MB of RAM which does not get reflected in the mprof plot. How can I plot the memory consumption of each Celery task triggered by the py.test?

P.S我使用内存探查器包来探查python脚本的内存消耗.

P.S I use memory profiler package to profile the memory consumption of a python script.

推荐答案

-multiprocess 标志如何? https://github.com/pythonprofilers/memory_profiler#tracking-forked-child-进程

这篇关于我们如何描述每个Celery任务的内存使用情况?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！