如何限制堆大小? [英] How to limit the heap size?

问题描述

我有时编写Python程序，这些程序很难确定执行前将使用多少内存.因此，有时我会调用一个Python程序，尝试分配大量的RAM，从而导致内核大量交换并降低其他正在运行的进程的性能.

I sometimes write Python programs which are very difficult to determine how much memory it will use before execution. As such, I sometimes invoke a Python program that tries to allocate massive amounts of RAM causing the kernel to heavily swap and degrade the performance of other running processes.

因此，我希望限制Python堆可以增长多少内存.达到限制后，程序可能会崩溃.最好的方法是什么?

Because of this, I wish to restrict how much memory a Python heap can grow. When the limit is reached, the program can simply crash. What's the best way to do this?

如果有关系，很多代码用Cython编写，因此应考虑在那里分配的内存.我还没有嫁给一个纯Python解决方案(它不需要是可移植的)，因此在Linux上运行的任何东西都可以.

If it matters, much code is written in Cython, so it should take into account memory allocated there. I am not married to a pure Python solution (it does not need to be portable), so anything that works on Linux is fine.

推荐答案

查看 resource.setrlimit().它只能在Unix系统上使用，但似乎可能正是您所需要的，因为您可以使用resource.RLIMIT_DATA参数为进程和进程的子进程选择最大堆大小.

Check out resource.setrlimit(). It only works on Unix systems but it seems like it might be what you're looking for, as you can choose a maximum heap size for your process and your process's children with the resource.RLIMIT_DATA parameter.

添加示例:

import resource

rsrc = resource.RLIMIT_DATA
soft, hard = resource.getrlimit(rsrc)
print 'Soft limit starts as  :', soft

resource.setrlimit(rsrc, (1024, hard)) #limit to one kilobyte

soft, hard = resource.getrlimit(rsrc)
print 'Soft limit changed to :', soft

我不确定您的用例到底是什么，但是您可能需要对堆栈大小进行限制，而不是使用resouce.RLIMIT_STACK.超过此限制将向您的进程发送一个SIGSEGV信号，并且要处理该信号，您将需要按照setrlimit Linux

I'm not sure what your use case is exactly but it's possible you need to place a limit on the size of the stack instead with resouce.RLIMIT_STACK. Going past this limit will send a SIGSEGV signal to your process, and to handle it you will need to employ an alternate signal stack as described in the setrlimit Linux manpage. I'm not sure if sigaltstack is implemented in python, though, so that could prove difficult if you want to recover from going over this boundary.

这篇关于如何限制堆大小?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！