IOError: [Errno 24] 打开的文件太多: [英] IOError: [Errno 24] Too many open files:

问题描述

我有一个巨大的文件，我正在写入大约 450 个文件.由于打开的文件太多，我收到错误消息.我在网上搜索并找到了一些解决方案，但没有帮助.

I have a huge file that I am writing into approximately 450 files. I am getting error as too many files open. I searched the web and found some solution but it is not helping.

import resource
resource.setrlimit(resource.RLIMIT_NOFILE, (1000,-1))
>>> len(pureResponseNames) #Filenames 
434
>>> resource.getrlimit(resource.RLIMIT_NOFILE)
(1000, 9223372036854775807)
>>> output_files = [open(os.path.join(outpathDirTest, fname) + ".txt", "w") for fname in pureResponseNames]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: [Errno 24] Too many open files: 'icd9_737.txt'
>>> 

我还从命令行更改了 ulimit，如下所示:

I also changed ulimit from the command line as below:

$ ulimit -n 1200
$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1200
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 709
virtual memory          (kbytes, -v) unlimited
$ 

我仍然遇到同样的错误.PS:我也重新启动了系统并运行了程序，但没有成功.

I am still getting the same error. PS: I also restarted my system and run the program but with no success.

推荐答案

打开的文件太多"错误总是很棘手——您不仅要摆弄 ulimit，还必须检查系统范围的限制和 OSX 特定的.这篇 SO 帖子提供了有关 OSX 中打开文件的更多信息.(剧透警报:默认为 256).

"Too many open files" errors are always tricky – you not only have to twiddle with ulimit, but you also have to check system-wide limits and OSX-specifics. This SO post gives more information on open files in OSX. (Spoiler alert: the default is 256).

但是，限制必须同时打开的文件数量通常很容易.如果我们看看 Stefan Bollman 的例子，我们可以轻松地将其更改为:

However, it is often easy to limit the number of files that have to be open at the same time. If we look at Stefan Bollman's example, we can easily change that to:

pureResponseNames = ['f'+str(i) for i in range(434)]
outpathDirTest="testCase/"
output_files = [os.path.join(outpathDirTest, fname) + ".txt" for fname in pureResponseNames]

for filename in range(output_files):
    with open(filename, 'w') as f:
        f.write('This is a test of file nr.'+str(i))

这篇关于IOError: [Errno 24] 打开的文件太多:的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！