h5py OSError:无法打开文件(未找到文件签名) [英] h5py OSError: Unable to open file (File signature not found)
问题描述
对于使用h5py时收到的错误，我有些困惑.我正在尝试应用python脚本来循环遍历位于不同目录中的h5py文件集.例如，第一组h5py文件位于
Reduced/rho = 0.75/2/Data/snapshots
当我运行 精简版子目录中的python脚本
时减少/test_h5py
使用以下python脚本
import h5py
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colors as colors
import cmocean
import os
de.logging_setup.rootlogger.setLevel('ERROR')
# Plot writes
path = '../rho=0.75/2/Data/snapshots'
for filename in os.listdir(path):
with h5py.File(path+'/'+filename,'r') as file:
一切正常，脚本遍历数据并提供输出.现在，当我尝试将相同的python脚本应用于其他数据时，即位于以下位置的h5py文件集，就会出现问题:
Reduced/rho = 0.75/4/Data/snapshots
现在，当我运行前面的python脚本时，只需对2到4的路径进行修改
import h5py
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colors as colors
import cmocean
import os
de.logging_setup.rootlogger.setLevel('ERROR')
# Plot writes
path = '../rho=0.75/4/Data/snapshots'
for filename in os.listdir(path):
with h5py.File(path+'/'+filename,'r') as file:
我收到以下错误
Traceback (most recent call last):
File "newest_edit.py", line 17, in <module>
with h5py.File(path+'/'+filename,'r') as file:
File "/usr/local/lib/python3.5/site-packages/h5py/_hl/files.py", line 269, in __init__
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "/usr/local/lib/python3.5/site-packages/h5py/_hl/files.py", line 99, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 78, in h5py.h5f.open
OSError: Unable to open file (file signature not found)
任何人都可以理解吗?我以为可能是os.listdir()错误，但搜索后没有找到任何解决问题的方法.感谢您的帮助.
我忘了提.当我在Jupyter笔记本中运行python脚本并将该脚本应用于单独的h5py文件时，它对于2和4路径都可以正常工作. h5py文件已打开，我可以检索存储在其上的数据，所以我无法想象它会是损坏的文件还是不是hdf5格式.
我真的不知道我在这里说的是什么，因此，如果这没有任何意义或帮助，或者与您无关，请提前对不起.
这是一个非常烦人的错误，一年多来没有人试图回答，所以我认为我所拥有的一小部分知识可能会有所帮助.
在集群上工作时，我遇到了类似的问题.事实证明，h5py使用称为flock()的东西，有时会对文件系统性能产生负面影响.
因为有时不允许使用此flock().如果您正在使用其他人的文件系统，请询问admin他们是否会降低它的权限，否则，我找不到解决方法(我没有尝试找到不使用flock()的其他库).
如果它是您自己的文件系统，那么您应该可以允许flock()，但是我不确定如何执行此操作. 此如果它是 NFS
我当然想听听你过得怎么样?
I'm a bit confused about an error I'm receiving when using h5py. I'm trying to apply a python script to loop through sets of h5py files located in different directories. For example, the first set of h5py files is located at
Reduced/rho=0.75/2/Data/snapshots
When I run the python script from a sub directory of Reduced
Reduced/test_h5py
with the following python script
import h5py
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colors as colors
import cmocean
import os
de.logging_setup.rootlogger.setLevel('ERROR')
# Plot writes
path = '../rho=0.75/2/Data/snapshots'
for filename in os.listdir(path):
with h5py.File(path+'/'+filename,'r') as file:
everything works fine, the script loops through the data and gives me an output. Now, the issue arises when I try to apply the same python script to other data i.e at the set of h5py files located at
Reduced/rho=0.75/4/Data/snapshots
Now, when I run the previous python script with just a modification in the path from 2 to 4
import h5py
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colors as colors
import cmocean
import os
de.logging_setup.rootlogger.setLevel('ERROR')
# Plot writes
path = '../rho=0.75/4/Data/snapshots'
for filename in os.listdir(path):
with h5py.File(path+'/'+filename,'r') as file:
I get the following error
Traceback (most recent call last):
File "newest_edit.py", line 17, in <module>
with h5py.File(path+'/'+filename,'r') as file:
File "/usr/local/lib/python3.5/site-packages/h5py/_hl/files.py", line 269, in __init__
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "/usr/local/lib/python3.5/site-packages/h5py/_hl/files.py", line 99, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 78, in h5py.h5f.open
OSError: Unable to open file (file signature not found)
Can anyone make sense of this? I thought it might be a os.listdir() error but after searching I didn't find anything to solve the issue. Thanks for your help.
EDIT:
I forgot to mention. When I run the python script in a Jupyter notebook and apply the script to individual h5py files it works just fine for both the 2 and 4 paths. The h5py files open and I can retrieve the data stored on them, so I can't imagine it would be corrupted files or not in hdf5 format.
I don't really know what I'm talking about here so sorry in advance if this doesn't make sense or help or just isn't related.
It's an extremely annoying error that no one has attempted to answer in over a year so I thought the tiny bit of knowledge I have might help.
I had a similar problem whilst working on a cluster. It turned out that h5py uses something called flock() and that this can sometimes have a negative effect on the filesystem performance.
Because of this flock() is sometimes not allowed. If you are working on someone elses filesystem then ask admin if they will alow it and if not then I can't find a way around it (I have not tried to find other libraries that don't use flock()).
If it is your own filesystem then you should be able to allow flock() but I'm not sure how to do this. This may help you if it's NFS
I would certainly be interested to hear how you got on?
这篇关于h5py OSError:无法打开文件(未找到文件签名)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
