如何在Python中使用psutil获得程序的最大内存使用率 [英] How to get the max memory usage of a program using psutil in Python
问题描述
我正在使用以下代码来获取程序的最大内存使用量.
I am using the following code to get the max memory usage of the program.
import os, subprocess , psutil
def mem(cmd):
try:
with open('in.txt','r') as infile, open('out.txt', 'w') as outfile:
p=psutil.Popen("./"+cmd,shell=False,stdin=infile,stdout = outfile)
print p.memory_info()
except Exception:
print "Error"
cmd=raw_input()
mem(cmd)
问题有时是在程序的初次运行时，内存使用量输出为(0,0)，但随后它显示正确的输出.我不知道为什么会这样.对于某些程序，例如c ++中的hello world程序，输出为pmem(rss=4096, vms=315392)，大约为0.3M(我认为输出以字节为单位)，但是在ideone.com中运行hello world程序则输出为〜3M.为什么会有这种分散性?
The problem is sometimes for initial runs of the program the memory usage output is (0,0) but subsequently it displays the correct output. I dont why this happening. For some programs such as the hello world program in c++ the output is pmem(rss=4096, vms=315392) which is about 0.3M( I think the output is in bytes ) but running the hello world program in ideone.com gives the output as ~3M. Why is there this disperency?
cmd是可执行文件的名称.
cmd is the name of the executable.
命令print subprocess.check_output(['ps', 'v', '-p', str(p.pid)])
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
16150 pts/16 Z+ 0:00 0 0 0 0 0.0 [a.out] <defunct>
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
16150 pts/16 Z+ 0:00 0 0 0 0 0.0 [a.out] <defunct>
我的示例C ++程序之一:
One of my sample C++ program:
`int a[1000000];
int main()
{
return 0;
}`
有时返回pmem(rss = 4096，vms = 4313088)，有时返回pmem(rss = 0，vms = 0)
returns pmem(rss=4096, vms=4313088) sometimes and pmem(rss=0,vms=0) sometimes
推荐答案
<defunct> means that the subprocess is a zombie process (it is dead but its status has not been read yet by the parent (p.poll() or p.wait())). It seems both psutil and ps shows RSS to be zero for such processes.
结果取决于子进程是否将在调用p.memory_info()之前退出.这是一场比赛.如果在C ++程序的出口处添加延迟，则p.memory_info()可能在子进程退出之前被调用，并且您应该获得非零的结果.
The result depends on whether the subprocess will exit sooner than p.memory_info() is called. It is a race. If you add a delay at the exit in the C++ program then p.memory_info() may be called before the subprocess exits and you should get non-zero results.
问题是我可以得到任意程序进行评估.语言也不是固定的.没有解决这个问题的好方法吗?
The problem is that I can get arbitrary programs to evaluate . The language is also not fixed. Isn't there an elegant solution to this problem?
您可能需要操作系统支持才能保存有关子进程的内存使用情况的信息，即使该子进程退出后也是如此.或者，您可以使用内存分析器(例如valgrind)运行该程序并读取其结果.要收集结果:
You might need OS support to save info about the memory usage by a subprocess even after it exits. Or you could run the program using a memory profiler such as valgrind and read its results. To collect results:
$ valgrind --tool=massif cmd arg1 arg2
要查看结果，可以使用ms_print:
To see the results, you could use ms_print:
$ ms_print massif.out.* | less
或GUI Massif-Visualizer
Or GUI Massif-Visualizer
建议使用@mdadm 一个更简单的解决方案:
@mdadm suggested a simpler solution: time command:
from subprocess import Popen, PIPE
p = Popen(['time', '-f', '%M'] + args, stderr=PIPE)
ru_maxrss = int(p.communicate()[1])
print("Maximum rss %d KB" % ru_maxrss)
GNU时间使用wait3()填充资源使用情况信息(如果可用).可以在Python中调用:
GNU time uses wait3() to populate resource usage info if it is available. It can be called in Python:
import os
from subprocess import Popen
p = Popen(args)
ru = os.wait4(p.pid, 0)[2]
print("Maximum rss %d KB" % ru.ru_maxrss)
我将psutil.Process.memory_info(rss)返回的最大值与os.wait4返回的ru_maxrss值以及valgrind --tool=massif报告的最大总内存进行了比较:它们相似
I've compared the maximum value returned by psutil.Process.memory_info (rss) with ru_maxrss value returned by os.wait4 and with the maximum total memory reported by valgrind --tool=massif: they are similar.
另请参阅:
- A way to determine a process's "real" memory usage, i.e. private dirty RSS?
- How is memory usage reported in Linux?
- Find maximum memory consumed by a process specified at the command-line.
这篇关于如何在Python中使用psutil获得程序的最大内存使用率的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
