向HDF5文件中添加新数据会导致一个空数组 [英] Adding new data into HDF5 file results an empty array

问题描述

在使用适用于Python的HDF5软件包时，我发现了一个奇怪的行为.我想在表中插入更多数据.但是不知何故我无法使其正常工作.正如您从源代码中看到的那样，我正在使用 fromRow = hf ["X"].shape [0] 获取键'X'中的最后一行数据，并编写了 tempArray2之后.结果是一个空表.

While playing with HDF5 package for Python I discovered a strange behavior. I want to insert more data into table. But somehow I cannot get it work properly. As you can see from the source code, I am getting the last row of data in key 'X' using fromRow = hf["X"].shape[0] and writing the tempArray2 afterwards. Result is an empty table.

import h5py

tempArray1 = [[0.9293237924575806, -0.32789671421051025, 0.18110771477222443], [0.9293237924575806, -0.32789671421051025, 0.18110771477222443], [0.9293237924575806, -0.32789671421051025, 0.18110771477222443], [0.9293237924575806, -0.32789671421051025, 0.18110771477222443], [0.9293237924575806, -0.32789671421051025, 0.18110771477222443], [0.9293237924575806, -0.32789671421051025, 0.18110771477222443], [0.9293237924575806, -0.32789671421051025, 0.18110771477222443], [0.9293237924575806, -0.32789671421051025, 0.18110771477222443], [0.9293237924575806, -0.32789671421051025, 0.18110771477222443], [0.9293237924575806, -0.32789671421051025, 0.18110771477222443]]
tempArray2 = [[3.1387749004352372e-06, 8.120089097236803e+27, -1.645612730013634e-14], [3.1387749004352372e-06, 8.120089097236803e+27, -1.645612730013634e-14], [3.1387749004352372e-06, 8.120089097236803e+27, -1.645612730013634e-14], [3.1387749004352372e-06, 8.120089097236803e+27, -1.645612730013634e-14], [3.1387749004352372e-06, 8.120089097236803e+27, -1.645612730013634e-14], [3.1387749004352372e-06, 8.120089097236803e+27, -1.645612730013634e-14], [3.1387749004352372e-06, 8.120089097236803e+27, -1.645612730013634e-14], [3.1387749004352372e-06, 8.120089097236803e+27, -1.645612730013634e-14], [3.1387749004352372e-06, 8.120089097236803e+27, -1.645612730013634e-14], [3.1387749004352372e-06, 8.120089097236803e+27, -1.645612730013634e-14]]

with h5py.File('data.hdf5', 'w') as hf:
    # Add data to new file
    dset = hf.create_dataset("X", data=tempArray1, compression="gzip", chunks=True, maxshape=(None,3), dtype='f4') # Size is as the size of tempArray1
    print(hf["X"].shape[0])

    # Append data existing file
    hf["X"].resize((hf["X"].shape[0] + 10, 3)) # Size is as the size of X+ 10
    print(hf["X"].shape[0])
    fromRow = hf["X"].shape[0]
    hf["X"][fromRow:] = tempArray2

这是它的外观:

Key: X
Data:
 [[ 0.9293238  -0.3278967   0.18110771]
 [ 0.9293238  -0.3278967   0.18110771]
 [ 0.9293238  -0.3278967   0.18110771]
 [ 0.9293238  -0.3278967   0.18110771]
 [ 0.9293238  -0.3278967   0.18110771]
 [ 0.9293238  -0.3278967   0.18110771]
 [ 0.9293238  -0.3278967   0.18110771]
 [ 0.9293238  -0.3278967   0.18110771]
 [ 0.9293238  -0.3278967   0.18110771]
 [ 0.9293238  -0.3278967   0.18110771]
 [ 0.          0.          0.        ]
 [ 0.          0.          0.        ]
 [ 0.          0.          0.        ]
 [ 0.          0.          0.        ]
 [ 0.          0.          0.        ]
 [ 0.          0.          0.        ]
 [ 0.          0.          0.        ]
 [ 0.          0.          0.        ]
 [ 0.          0.          0.        ]
 [ 0.          0.          0.        ]]
Length of data: 20

当我用数字10替换 fromRow 的值(如 fromRow = 10 ，它代表现有表的末尾)时，就足够了.

Strangely enough when I replace the value fromRow with number 10 like fromRow = 10, which represents the end of the existing table, it works.

输出:

Key: X
Data:
 [[ 9.2932379e-01 -3.2789671e-01  1.8110771e-01]
 [ 9.2932379e-01 -3.2789671e-01  1.8110771e-01]
 [ 9.2932379e-01 -3.2789671e-01  1.8110771e-01]
 [ 9.2932379e-01 -3.2789671e-01  1.8110771e-01]
 [ 9.2932379e-01 -3.2789671e-01  1.8110771e-01]
 [ 9.2932379e-01 -3.2789671e-01  1.8110771e-01]
 [ 9.2932379e-01 -3.2789671e-01  1.8110771e-01]
 [ 9.2932379e-01 -3.2789671e-01  1.8110771e-01]
 [ 9.2932379e-01 -3.2789671e-01  1.8110771e-01]
 [ 9.2932379e-01 -3.2789671e-01  1.8110771e-01]
 [ 3.1387749e-06  8.1200891e+27 -1.6456127e-14]
 [ 3.1387749e-06  8.1200891e+27 -1.6456127e-14]
 [ 3.1387749e-06  8.1200891e+27 -1.6456127e-14]
 [ 3.1387749e-06  8.1200891e+27 -1.6456127e-14]
 [ 3.1387749e-06  8.1200891e+27 -1.6456127e-14]
 [ 3.1387749e-06  8.1200891e+27 -1.6456127e-14]
 [ 3.1387749e-06  8.1200891e+27 -1.6456127e-14]
 [ 3.1387749e-06  8.1200891e+27 -1.6456127e-14]
 [ 3.1387749e-06  8.1200891e+27 -1.6456127e-14]
 [ 3.1387749e-06  8.1200891e+27 -1.6456127e-14]]
Length of data: 20

知道我在做什么错吗?

推荐答案

在调整X数据集的大小之后，您将获得 fromRow .在调整大小之前，您需要该值.参见下面的代码.

You are getting fromRow AFTER you resize the X dataset. You need the value BEFORE your resize. See code below.

with h5py.File('data.hdf5', 'w') as hf:
    # Add data to new file
    dset = hf.create_dataset("X", data=tempArray1, compression="gzip", chunks=True, maxshape=(None,3), dtype='f4') # Size is as the size of tempArray1
    print(hf["X"].shape[0])
# new location to get fromRow:
    fromRow = hf["X"].shape[0]

    # Append data existing file
    hf["X"].resize((hf["X"].shape[0] + 10, 3)) # Size is as the size of X+ 10
    print(hf["X"].shape[0])        
    hf["X"][fromRow:] = tempArray2

这篇关于向HDF5文件中添加新数据会导致一个空数组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！