如何使用h5py编辑h5文件? [英] How to edit h5 files with h5py?

问题描述

有关使用h5py覆盖数组的问题未能解决我的问题. 我要编辑VGG16模型的数组值.

The question on overwrite array using h5py did not solve my problem. I want to edit the array values of a VGG16 model.

f = h5py.File('C:/Users/yash/.keras/models/vgg16_weights_tf_dim_ordering_tf_kernels_2.h5', mode = 'a')
ab = list(h5py.AttributeManager.keys(f))
print(list(f.attrs.keys()))
print(ab)

上面的代码返回:

['layer_names']


['block1_conv1', 'block1_conv2', 'block1_pool', 'block2_conv1', 'block2_conv2', 'block2_pool', 'block3_conv1', 'block3_conv2', 'block3_conv3', 
'block3_pool', 'block4_conv1', 'block4_conv2', 'block4_conv3', 'block4_pool',
'block5_conv1', 'block5_conv2', 'block5_conv3', 'block5_pool', 'fc1', 'fc2', 

'flatten', 'predictions']

使用此代码后: print(f.attrs['layer_names'])

我得到以下信息:

[b'block1_conv1' b'block1_conv2' b'block1_pool' b'block2_conv1'
 b'block2_conv2' b'block2_pool' b'block3_conv1' b'block3_conv2'
 b'block3_conv3' b'block3_pool' b'block4_conv1' b'block4_conv2'
 b'block4_conv3' b'block4_pool' b'block5_conv1' b'block5_conv2'
 b'block5_conv3' b'block5_pool' b'flatten' b'fc1' b'fc2' b'predictions']

如何更改f.attrs['layer_names']中包含的值?我无法编辑它们，主要是因为使用了: print(f.attrs['layer_names/block1_conv1'])返回错误.

How can i change the values that are contained within the f.attrs['layer_names']? I am not able to edit them mainly because using: print(f.attrs['layer_names/block1_conv1']) returns an error.

每个块(n)_conv(n)内都有一个权重和偏差矩阵.

There is a weight and bias matrix inside every block(n)_conv(n).

我想更改这些值.

我正在python 3中执行此操作，并且没有文档帮助我编辑这些值.主要是因为如果不使用此代码，我将无法访问这些文件:

I am doing this in python 3, and no documentation helped me in editing these values. Mostly because i am unable to access these without using this code:

layer = h5py.AttributeManager.get(f, key = str(layerstringlist[i]))
 nplayer = np.asarray(list(layer))

layerstringlist是这种方式的列表:

['block1_conv1/block1_conv1_W_1:0', 'block1_conv1/block1_conv1_b_1:0', .....
'predictions/predictions_W_1:0', 'predictions/predictions_b_1:0']

这可以正确返回它，但是我无法保存修改后的h5文件，因为我不知道如何在python 3中引用它.

This returns it correctly, but i am unable to save the modified h5 file because i do not know how to reference it in python 3.

提前谢谢！

推荐答案

我以前从未见过AttributeManager的使用，可能是因为文档不鼓励使用AttributeManager

I haven't seen the use of AttributeManager before, perhaps because the documentation discourages its use, http://docs.h5py.org/en/latest/high/attr.html#reference

有了其他SO测试遗留的文件，我得到了:

With a file left over from other SO tests I get:

In [480]: list(h5py.AttributeManager.keys(f))
Out[480]: ['agroup', 'agroup1', 'agroup2', 'arr']
In [481]: list(f.attrs.keys())
Out[481]: []
In [482]: list(f.keys())
Out[482]: ['agroup', 'agroup1', 'agroup2', 'arr']

在这种情况下，我没有为文件分配任何属性，因此f.attrs.keys()为空.您的文件似乎具有一个属性"layer_names".它的值是一个名称列表，您可以使用print(f.attrs['layer_names'])列出该名称.

In this case I have not assigned any attributes to the file, hence f.attrs.keys() is empty. It appears that your file has one attribute, 'layer_names'. Its value is a list of names, which you list with print(f.attrs['layer_names']).

AttributeManager列出组和数据集，而不是attrs.我在f.keys()上得到了相同的列表.

The AttributeManager lists the groups and datasets, not the attrs. I get the same list with f.keys().

您应该通过以下方式访问这些组或数据集之一:

You should be access one of these groups or datasets with:

f['block1_conv1']

如果这是一个组，则需要向下索引另一层.如果它是数据集，请按照

If this is a group you need to index down another layer. If it is a dataset, read and write to it as described in http://docs.h5py.org/en/latest/high/dataset.html#reading-writing-data

我认为f.attrs['layer_names']列表对您没有任何用处，因为它具有与`list(f.keys())相同的信息.

I don't think the f.attrs['layer_names'] list is of any use to you, since it has the same information as `list(f.keys()).

根据您的评论，f['block1_conv1']是一个小组，其中包含几个数据集.这些是索引集合的等效方法:

Based on your comment, f['block1_conv1'] is a group, with contains several datasets. These are equivalent ways of indexing a set:

f['block1_conv1/block1_conv1_W_1:0'] 
f['block1_conv1']['block1_conv1_W_1:0']

在我的测试文件中

In [483]: f['arr']
Out[483]: <HDF5 dataset "arr": shape (3,), type "|V31">

我可以使用value或[:]将数据集作为数组加载到内存中:

I can load the dataset into memory as an array with value or [:]:

In [485]: f['arr'].value
Out[485]: 
array([(123, 1, 1, 1, 1, 1, 1, 1), (  1, 1, 1, 1, 1, 1, 1, 1),
       (  1, 1, 1, 1, 1, 1, 1, 1)],
      dtype=[('Status', '<u8'), ('Segments', '<u4'), ('Characterized', '<u4'), ('More_Segments', '<u4'), ('ID', '<i4'), ('Releases', '<u2'), ('Type', 'u1'), ('Track', '<i4')])
In [486]: f['arr'][:]
Out[486]: 
array([(123, 1, 1, 1, 1, 1, 1, 1), (  1, 1, 1, 1, 1, 1, 1, 1),
       (  1, 1, 1, 1, 1, 1, 1, 1)],
      dtype=[('Status', '<u8'), ('Segments', '<u4'), ('Characterized', '<u4'), ('More_Segments', '<u4'), ('ID', '<i4'), ('Releases', '<u2'), ('Type', 'u1'), ('Track', '<i4')])

(对不起，此示例是一个复杂的结构化数组.)

(sorry, this example is a complicated structured array.)

我可以修改此数据集的值，就像修改相同类型和形状的数组一样

I can modify the values of this dataset just as I would modify an array of the same type and shape

In [487]: f['arr']['Status']
Out[487]: array([123,   1,   1], dtype=uint64)
In [488]: f['arr']['Status'] = [1,2,3]

我无法替换它. f['arr'] = np.arange(10)给我一个错误(名称已存在). f['arr'][:] = np.arange(10)给出了不同的错误(关于不兼容的形状).

I cannot replace it. f['arr'] = np.arange(10) gives me an error (name already exists). f['arr'][:] = np.arange(10) gives a different error (about incompatible shapes).

我可以使用其他名称创建一个新的数据集

I could create a new dataset with a different name

In [492]: f.create_dataset('newarray', np.arange(10))
Out[492]: <HDF5 dataset "newarray": shape (0, 1, 2, 3, 4, 5, 6, 7, 8, 9), type "<f4">
In [493]: list(f.keys())
Out[493]: ['agroup', 'agroup1', 'agroup2', 'arr', 'newarray']

我可以使用以下方法删除数据集:

I can delete a dataset with:

In [494]: del f['newarray']
In [495]: list(f.keys())
Out[495]: ['agroup', 'agroup1', 'agroup2', 'arr']

并使用以下名称定义一个新名称:

and define a new one with same name with:

In [500]: f.create_dataset('newarray', data=np.ones((3,4)))
Out[500]: <HDF5 dataset "newarray": shape (3, 4), type "<f8">

这篇关于如何使用h5py编辑h5文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！