Python subprocess.Popen错误，出现OSError:[Errno 12]在一段时间后无法分配内存 [英] Python subprocess.Popen erroring with OSError: [Errno 12] Cannot allocate memory after period of time
问题描述
Note: This question has been re-asked with a summary of all debugging attempts here.
我有一个Python脚本，它每60秒执行一次，作为后台进程运行.其中一部分是对 subprocess.Popen 的调用，以获取 ps .
I have a Python script that is running as a background process executing every 60 seconds. Part of that is a call to subprocess.Popen to get the output of ps.
ps = subprocess.Popen(['ps', 'aux'], stdout=subprocess.PIPE).communicate()[0]
运行几天后，呼叫出现错误:
After running for a few days, the call is erroring with:
File "/home/admin/sd-agent/checks.py", line 436, in getProcesses
File "/usr/lib/python2.4/subprocess.py", line 533, in __init__
File "/usr/lib/python2.4/subprocess.py", line 835, in _get_handles
OSError: [Errno 12] Cannot allocate memory
但是，服务器上免费的输出是:
However the output of free on the server is:
$ free -m
total used free shared buffers cached
Mem: 894 345 549 0 0 0
-/+ buffers/cache: 345 549
Swap: 0 0 0
I have searched around for the problem and found this article which says:
解决方案是为服务器添加更多的交换空间.当内核分叉启动建模器或发现过程时，它首先确保在交换存储中有足够的空间来存储新过程(如果需要).
Solution is to add more swap space to your server. When the kernel is forking to start the modeler or discovery process, it first ensures there's enough space available on the swap store the new process if needed.
我注意到上面的免费输出中没有可用的交换.这可能是问题所在，并且/或者还有其他解决方案吗?
I note that there is no available swap from the free output above. Is this likely to be the problem and/or what other solutions might there be?
更新09年8月13日作为一系列监视功能的一部分，上述代码每60秒被调用一次.进程被守护，并使用 sched 安排检查.上述功能的具体代码为:
Update 13th Aug 09 The code above is called every 60 seconds as part of a series of monitoring functions. The process is daemonized and the check is scheduled using sched. The specific code for the above function is:
def getProcesses(self):
self.checksLogger.debug('getProcesses: start')
# Memory logging (case 27152)
if self.agentConfig['debugMode'] and sys.platform == 'linux2':
mem = subprocess.Popen(['free', '-m'], stdout=subprocess.PIPE).communicate()[0]
self.checksLogger.debug('getProcesses: memory before Popen - ' + str(mem))
# Get output from ps
try:
self.checksLogger.debug('getProcesses: attempting Popen')
ps = subprocess.Popen(['ps', 'aux'], stdout=subprocess.PIPE).communicate()[0]
except Exception, e:
import traceback
self.checksLogger.error('getProcesses: exception = ' + traceback.format_exc())
return False
self.checksLogger.debug('getProcesses: Popen success, parsing')
# Memory logging (case 27152)
if self.agentConfig['debugMode'] and sys.platform == 'linux2':
mem = subprocess.Popen(['free', '-m'], stdout=subprocess.PIPE).communicate()[0]
self.checksLogger.debug('getProcesses: memory after Popen - ' + str(mem))
# Split out each process
processLines = ps.split('\n')
del processLines[0] # Removes the headers
processLines.pop() # Removes a trailing empty line
processes = []
self.checksLogger.debug('getProcesses: Popen success, parsing, looping')
for line in processLines:
line = line.split(None, 10)
processes.append(line)
self.checksLogger.debug('getProcesses: completed, returning')
return processes
这是一个更大的类(称为check)的一部分，该类在守护程序启动时初始化一次.
This is part of a bigger class called checks which is initialised once when the daemon is started.
可以在 http://github上找到整个支票类.com/dmytton/sd-agent/blob/82f5ff9203e54d2adeee8cfed704d09e3f00e8eb/checks.py ，其具有从442行定义的getProcesses函数.doChecks()从520行开始调用.
The entire checks class can be found at http://github.com/dmytton/sd-agent/blob/82f5ff9203e54d2adeee8cfed704d09e3f00e8eb/checks.py with the getProcesses function defined from line 442. This is called by doChecks() starting at line 520.
推荐答案
在使用popen时，如果希望它关闭额外的文件描述符，则需要上交close_fds = True.
when you use popen you need to hand in close_fds=True if you want it to close extra file descriptors.
创建一个新管道，该管道发生在追溯跟踪的_get_handles函数中，会创建2个文件描述符，但是您当前的代码永远不会关闭它们，最终会达到系统的最大fd限制.
creating a new pipe, which occurs in the _get_handles function from the back trace, creates 2 file descriptors, but your current code never closes them and your eventually hitting your systems max fd limit.
不确定为什么收到的错误表示内存不足的情况:它应该是文件描述符错误，因为返回值pipe()包含针对此问题的错误代码.
Not sure why the error you're getting indicates an out of memory condition: it should be a file descriptor error as the return value of pipe() has an error code for this problem.
这篇关于Python subprocess.Popen错误，出现OSError:[Errno 12]在一段时间后无法分配内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
