Python-OSError 24(打开的文件过多)和共享内存 [英] Python - OSError 24 (Too many open files) and shared memory

问题描述

我遇到的问题是，在python脚本中的mac os x上引发了OSError 24异常(打开的文件太多").

I faced with the problem there was exception OSError 24 ("Too many open files") raised on my mac os x in python script.

我不知道是什么原因导致了这个问题. lsof -p显示大约40-50行，而我的ulimit为1200(我检查是否使用

I had no idea what could caused that issue. lsof -p showed about 40-50 lines, and my ulimit was 1200 (I check that using

resource.getrlimit(resource.RLIMIT_NOFILE)

)，返回元组(1200，1200).所以我什至没有超过极限.

), that returned tuple (1200, 1200). So I didn't exceed limit even closely.

我的脚本产生了许多子进程，并且还分配了共享内存段.分配共享内存段时发生异常.我使用 sysv_ipc模块.

That my script spawned number of subprocesses and also allocated shared memory segments. Exception occurred while allocating shared memory segments. I use sysv_ipc module.

我还知道我允许的共享内存段总数足够大(128个段)，并且可以执行命令

Also I knew I total allowed number of shared memory segments is enough large (128 segments), and command

ipcs -b -m

绝对减少数量(不超过40个细分).

gave definitely less number (not more then 40 segments).

推荐答案

问题出在共享内存系统设置中(共享内存– Wiki ).

The issue was in shared memory system settings (shared memory – wiki).

/etc/sysctl.conf文件中有一个参数kern.sysv.shmseg，它表示每个进程可以附加的共享内存段的最大数目.因此，我只有32个值，不足以满足我的脚本要求.

There is parameter kern.sysv.shmseg in /etc/sysctl.conf file which represents the maximum number of shared memory segments each process can attach. So I had value 32 that was not enough for my script.

要查看参数，请使用:

sysctl -A | grep shm

要更新该参数，请编辑文件:

To update that parameters, edit file:

sudo vim /etc/sysctl.conf

我现在看起来像这样:

kern.sysv.shmmax=564777216
kern.sysv.shmmin=1
kern.sysv.shmmni=700
kern.sysv.shmseg=128
kern.sysv.shmall=131072

注意，您需要重新启动系统才能应用设置.

Notice, you need restart system in order to apply settings.

要查看当前分配的共享内存段，请输入:

To view currently allocated shared memory segments, type:

ipcs -m -b

要删除所有共享内存段，请执行以下操作:

To remove all shared memory segments:

for n in `ipcs -b -m | egrep ^m | awk '{ print $2; }'`; do ipcrm -m $n; done

请注意，只有未附加到任何进程的段才会被真正删除.

Notice, only segments that are not attached to any process will be really removed.

有关共享内存设置的更多信息: http://techjournal.318 .com/general-technology/shared-memory-settings-explain/， http://www.spy-hill.com/help/apple/SharedMemory.html ， http://support.apple.com/kb/HT4022

More on shared memory settings: http://techjournal.318.com/general-technology/shared-memory-settings-explain/, http://www.spy-hill.com/help/apple/SharedMemory.html, http://support.apple.com/kb/HT4022

这篇关于Python-OSError 24(打开的文件过多)和共享内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！