如何压缩保存在hdf5中的数据? [英] How to compress the data that saved in hdf5?

问题描述

我正在使用python 2.7读取视频并将其存储在hdf5中.这是我的代码

I am using python 2.7 to read a video and store in hdf5. This is my code

import h5py
import skvideo.datasets
import skvideo.io
videodata = skvideo.io.vread('./v_ApplyEyeMakeup_g01_c01.avi')
with h5py.File('./video.hdf5','w') as f:
    f['data'] = videodata
    f['label'] = 1

问题是输出hdf5太大.它比原始avi文件大128倍.如何压缩/减小尺寸?您可以从 https://drive.google.com/open?id=0B1MrjZsURl2yNFM0ZTJfZ3pOZVU /a>

The problem is that the output hdf5 is too larger. It is 128 times larger than the original avi file. What should I do to compress/reduce the size? You can download the file at https://drive.google.com/open?id=0B1MrjZsURl2yNFM0ZTJfZ3pOZVU

我认为我们可以使用

f.create_dataset('data',data=videodata,compression='gzip',compression_opts=9)
f.create_dataset('label', data=1)

现在，它仍然比原始文件大37倍.提前致谢.

Now, it still 37 times larger than the original file. Thanks in advance.

推荐答案

通过添加分块，我能够使输出为7.2M，而没有该功能则为10M.因此，它肯定会有所改善，但距离专用视频格式还差得远.您可以使用来自 https://support.hdfgroup.org/services/filters.html的其他过滤器，但我怀疑它们会将压缩程度提高一个数量级.因此，如果要继续使用h5py，则可能需要接受更大的文件大小.如果不能接受，请尝试其他文件格式.

By adding chunking I was able to make the output 7.2M compared to 10M without. So it definitely improves, but still far from dedicated video formats. You may play with other filters from https://support.hdfgroup.org/services/filters.html but I doubt they will improve the compression by an order of magnitude. So if you want to continue with h5py, you probably need to accept larger file size. In case this is not acceptable, just try another file format.

import h5py
import skvideo.datasets
import skvideo.io
videodata = skvideo.io.vread('./v_ApplyEyeMakeup_g01_c01.avi')

print(videodata.shape)
with h5py.File('./video.hdf5','w') as f:
    f.create_dataset('data',
                      data=videodata,
                      compression='gzip',
                      compression_opts=9,
                      chunks=(164, 20, 20, 3))
    f.create_dataset('label', data=1)

这篇关于如何压缩保存在hdf5中的数据?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！