使用astropy.io读取一堆FITS时出现OSError 24(打开的文件太多) [英] OSError 24 (Too many open files) when reading bunch of FITS with astropy.io

问题描述

我正在尝试使用astropy.io.fits将一些2000 FITS加载到内存中:

I’m trying to load into memory a few 2 000 FITS using astropy.io.fits:

def readfits(filename):
    with fits.open(filename) as ft:
        # the fits contain a single HDU
        data = ft[0].data
    return data

data_sci = []
for i in range(2000):
    data_sci.append(readfits("filename_{}.fits".format(i)))

但是，当到达第1015个文件时，会引发OSError: [Errno 24] Too many open files.

However, when reaching the 1015th file, OSError: [Errno 24] Too many open files is raised.

我对以下问题也有同样的看法:

I have the same issue with:

def readfits(filename):
    ft = fits.open(filename) as ft:
    data = ft[0].data
    ft.close()
    return data

我怀疑astropy.io.fits无法正确关闭文件.有没有 我可以强制关闭文件吗?

I suspect that astropy.io.fits does not properly close the file. Is there a way I can force the files to be closed?

推荐答案

您的readfits函数实际上使文件句柄保持打开状态，以便保持对数据的访问，因为默认情况下，它会创建 http://astropy.readthedocs.org/en/latest/io/fits/appendix/faq.html#im-opening-many-fits-files-in-a-loop-and-getting-oserror-太多打开文件

Your readfits function actually leaves the file handle open in order to keep access to the data, because by default it creates a mmap to the data and does not read it entirely into physical memory, as explained: http://astropy.readthedocs.org/en/latest/io/fits/appendix/faq.html#i-m-opening-many-fits-files-in-a-loop-and-getting-oserror-too-many-open-files

顺便说一句，如果您只想从第一个HDU中读取数据的函数，则该函数已经内置:

Incidentally, if you just want a function that reads the data out of the first HDU this is already built in: http://docs.astropy.org/en/v1.0.5/io/fits/api/files.html#astropy.io.fits.getdata

没有必要重新发明轮子.

It's not necessary to reinvent the wheel.

这篇关于使用astropy.io读取一堆FITS时出现OSError 24(打开的文件太多)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！