HDF5:如何将数据追加到数据集(可扩展数组) [英] HDF5: How to append data to a dataset (extensible array)

问题描述

按照本教程，我试图扩展我的HDF5数据集.代码如下，但是数据未正确写入数据集(数据集具有适当的最终大小，但仅包含零).与本教程的唯一区别在于，我必须使用动态数组.有什么主意吗?

By following this tutorial, I've tried to extend my HDF5 dataset. The code is the following, however the data is not properly written to the dataset (the dataset has the proper final size but contains only zeros). The only difference from the tutorial is that I have to use dynamic arrays. Any idea?

int main()
{    
    hsize_t dims[1], max_dims[1], newdims[1], chunk_dims[1], offset[1];
    hid_t file, file_space, plist, dataset, mem_space;
    int32_t *buffer1, *buffer2;

    file = H5Fcreate("test.h5", H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT);

    // Create dataspace with initial dim = 0 and final = UNLIMITED
    dims[0] = 0;
    max_dims[0] = H5S_UNLIMITED;
    file_space = H5Screate_simple(RANK, dims, max_dims);

    // Create dataset creation property list to have chunks
    plist = H5Pcreate(H5P_DATASET_CREATE);
    H5Pset_layout(plist, H5D_CHUNKED);
    chunk_dims[0] = 2;
    H5Pset_chunk(plist, RANK, chunk_dims);

    // Create the dataset
    dataset = H5Dcreate(file, "Test", H5T_NATIVE_INT32, file_space, H5P_DEFAULT, plist, H5P_DEFAULT);

    H5Pclose(plist);
    H5Sclose(file_space);

    //## FIRST BUFFER

    int length = 6;
    buffer1 = new int32_t[length];
    for (hsize_t i = 0; i < length; i++)
        buffer1[i] = i;

    // Extend the dataset by getting previous size and adding current length
    file_space = H5Dget_space(dataset);
    H5Sget_simple_extent_dims(file_space, dims, NULL);
    newdims[0] = dims[0] + length;
    H5Dset_extent(dataset, newdims);

    // Select hyperslab on the file dataset
    offset[0] = dims[0];
    dims[0] = length;
    H5Sselect_hyperslab(file_space, H5S_SELECT_SET, offset, NULL, dims, NULL);

    // Dataspace for buffer in memory
    mem_space = H5Screate_simple(RANK, dims, NULL);

    // Append buffer to dataset
    H5Dwrite(dataset, H5T_NATIVE_INT32, mem_space, file_space, H5P_DEFAULT, buffer1);

    H5Sclose(file_space);
    H5Sclose(mem_space);

    //## SECOND BUFFER

    length = 4;
    buffer2 = new int32_t[length];
    for (hsize_t i = 0; i < length; i++)
        buffer2[i] = i;

    // Extend the dataset by getting previous size and adding current length
    file_space = H5Dget_space(dataset);
    H5Sget_simple_extent_dims(file_space, dims, NULL);
    newdims[0] = dims[0] + length;
    H5Dset_extent(dataset, newdims);

    // Select hyperslab on the file dataset
    offset[0] = dims[0];
    dims[0] = length;
    H5Sselect_hyperslab(file_space, H5S_SELECT_SET, offset, NULL, dims, NULL);

    // Dataspace for buffer in memory
    mem_space = H5Screate_simple(RANK, dims, NULL);

    // Append buffer to dataset
    H5Dwrite(dataset, H5T_NATIVE_INT32, mem_space, file_space, H5P_DEFAULT, buffer2);

    H5Sclose(file_space);
    H5Sclose(mem_space);

    H5Dclose(dataset);
    H5Fclose(file);

   delete[] buffer1;
   delete[] buffer2;
}

推荐答案

我已经找到了解决问题的方法.它与动态数组无关.问题在于，在调用 H5Sget_simple_extent_dims 之后，数据空间ID会以某种方式失效(我不明白为什么...一个错误?)，您需要在重新使用它之前再次获取它，例如在选择超级平板之前:

I've figured out how to solve the problem. It has nothing to do with the dynamic arrays. The problem is that after calling H5Sget_simple_extent_dims the dataspace id gets somehow invalidated (I do not understand why...a bug?) and you need to get it again before reusing it, e.g. before selecting the hyperslab:

// Select hyperslab on the file dataset
offset[0] = dims[0];
dims[0] = length;
H5Dclose(file_space); // --ADDED-- CLOSE THE PREVIOUSLY OPENED
file_space = H5Dget_space(dataset); // --ADDED-- REOPEN
H5Sselect_hyperslab(file_space, H5S_SELECT_SET, offset, NULL, dims, NULL);

这篇关于HDF5:如何将数据追加到数据集(可扩展数组)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！