在SVHN数据集中访问hdf5组的两种方式之间有什么区别？ [英] What is the difference between the two ways of accessing the hdf5 group in SVHN dataset?

问题描述

我需要读取SVHN数据集，并试图读取第一个图像的文件名。

I need to read the SVHN dataset and was trying to read the filename of the first image.

我很难理解HDF5的结构，尤其是在理解SVHN数据集的层次结构方面

I am struggling a bit to understand the structure of HDF5 and especially in understanding the hierarchy/structure of the SVHN dataset

这两种读取图像名称的方法有什么区别？

What is the difference between these two approaches of reading the name of the image?

我在<$ c $的定义中遇到了此脚本中的方法1 c> getName（）函数： https： //github.com/bdiesel/tensorflow-svhn/blob/master/digit_struct.py

我在玩hdf5格式文件并想出了方法2尝试显示相同结果的不同事物。

I played around with the hdf5 format file and came up with method 2 while trying out different things that showed the same result.

# Both these methods read the first character of the name of the 1st
# image in svhn dataset
f = h5py.File(path_to_svhn_dataset,'r')

# method 1 
f[f['digitStruct']['name'][0][0]].value

# method 2
f[f['digitStruct']['name'].value[0].item()].value[0][0]

第一个图像是文件名为 1.png的文件。上面提到的两种获取文件名第一个字符的方式都将给我们int等效于ascii'1'-> 49

The first image is the file with filename "1.png". Both the above mentioned ways of getting the first character of the filename will give us int equivalent of ascii '1'-> 49

推荐答案

首先，这2种方法的输出存在细微差别。

方法1：返回（编码文件名的）完整数组

方法2：仅返回第一个数组的元素（字符）

First, there is a minor difference in output from your 2 methods.
Method 1: returns the full array (of the encoded file name)
Method 2: only returns the first element (character) of the array

让我们解构代码以了解您拥有的内容。

第一部分处理 h5py 数据对象。

Let's deconstruct your code to understand what you have.
The first part deals with h5py data objects.

f ['digitStruct'] ->返回h5py 组对象

f ['digitStruct'] ['name'] ->返回h5py 数据集对象

f ['digitStruct'] ['name']。name ->返回数据集对象的名称（路径）

f['digitStruct'] -> returns a h5py group object
f['digitStruct']['name'] -> returns a h5py dataset object
f['digitStruct']['name'].name -> returns the name (path) of the dataset object

注意：

/ digitStruct / name 数据集包含对象引用。每个数组条目都是一个指向另一个h5py对象（在本例中为另一个数据集）的指针。
例如（用于描绘2个对象引用的空格）：

f [f ['digitStruct'] ['name'] [0] [0]] ->返回在[0] [0]

处引用的对象因此，外部 f [obj_ref] 起作用像其他对象引用一样。

Note:
The /digitStruct/name dataset contains "Object References". Each array entry is a pointer to another h5py object (in this case another dataset). For example (spaces used to delineate the 2 object references):
f[ f['digitStruct']['name'][0][0] ] -> returns the object referenced at [0][0]
So, the outer f[ obj_ref ] works just like other object references.

对于 f ['digitStruct'] ['name'] [0] [0] ，这是一个指向数据集 /＃refs＃/ b
的对象，换句话说， f ['digitStruct'] ['name' ] [0] [0] 引用与以下对象相同的对象：
f ['＃refs＃'] ['b'] 或 f ['/＃refs＃/ b']

In the case of f['digitStruct']['name'][0][0], this is an object pointing to dataset /#refs#/b In other words, f['digitStruct']['name'][0][0] references the same object as: f['#refs#']['b'] or f['/#refs#/b']

对于h5py对象引用来说太多了。

让我们继续使用方法1 从该对象引用中获取数据。

So much for h5py object references.
Let's continue to get the data from this object reference using Method 1.

f [f [ 'digitStruct'] ['name'] [0] [0]]。value ->返回整个 /＃refs＃/ b 数据集作为一个NumPy数组。

f[f['digitStruct']['name'][0][0]].value -> returns the entire /#refs#/b dataset as a NumPy array.

但是，不建议使用 dataset.value ，而首选使用NumPy索引，如下所示：
f [f ['digitStruct'] ['name'] [0] [0]] [：] （获取整个数组）

However, dataset.value is deprecated, and NumPy indexing is preferred, like this: f[f['digitStruct']['name'][0][0]][:] (to get the entire array)

注意：这两个都返回整个编码字符数组。
此时，获得的名称是Python和NumPy功能。
使用此命令以字符串形式返回文件名：

f [f ['digitStruct'] ['name'] [0] [0]] [：] .tostring（）。decode（'ascii'）

Note: both of these return the entire array of encoded characters. At this point, getting the name is Python and NumPy fuctionality. Use this to return the filename as a string:
f[f['digitStruct']['name'][0][0]][:].tostring().decode('ascii')

现在，让我们解构用于方法2 的对象引用>。

Now let's deconstruct the object reference you used for Method 2.

f ['digitStruct'] ['name']。value
->返回整个 / digitStruct / name 数据集作为NumPy数组。
它有13,068行带有对象引用

f['digitStruct']['name'].value -> returns the entire /digitStruct/name dataset as a NumPy array. It has 13,068 rows with object references

f ['digitStruct'] ['name']。value [0] ->是第一行

f ['digitStruct'] ['name']。value [0] .item （）->将该数组元素复制到python标量

f['digitStruct']['name'].value[0].item() -> copies that array element to a python scalar

所有这些都指向同一个对象：

方法1： f ['digitStruct'] ['name'] [0] [0]

方法2： f ['digitStruct'] ['name']。value [0] .item（）

而且都与 f ['＃ refs＃'] ['b'] 或 f ['/＃refs＃/ b'] 。

So all of these point to the same object:
Method 1: f['digitStruct']['name'][0][0]
Method 2: f['digitStruct']['name'].value[0].item()
And are both the same as f['#refs#']['b'] or f['/#refs#/b'] for this example.

类似于方法1，获得的字符串是Python和NumPy功能。

Like Method 1, getting the string is Python and NumPy fuctionality.

f [f ['digitStruct '] ['name']。value [0] .item（）] [：]。tostring（）。decode（'ascii'）

是的，对象引用很复杂。...

我的建议：

使用NumPy索引而不是 .value从对象中提取NumPy数组（如上面的修改方法1所示）。

Yes, object references are complicated....
My recommendation:
Extract NumPy arrays from objects using NumPy indexing instead of .value (as shown in Modified Method 1 above).

示例完整性代码。

import h5py

# Both of these methods read the name of the 1st
# image in svhn dataset
f = h5py.File('test_digitStruct.mat','r')
print (f['digitStruct'])
print (f['digitStruct']['name'])
print (f['digitStruct']['name'].name)

# method 1
print('\ntest method 1')
print (f[f['digitStruct']['name'][0][0]])
print (f[f['digitStruct']['name'][0][0]].name)
#  both of these get the entire array / filename:
print (f[f['digitStruct']['name'][0][0]].value)
print (f[f['digitStruct']['name'][0][0]][:]) # same as .value above
print (f[f['digitStruct']['name'][0][0]][:].tostring().decode('ascii'))

# method 2
print('\ntest method 2')
print (f[f['digitStruct']['name'].value[0].item()]) 
print (f[f['digitStruct']['name'].value[0].item()].name) 

# this only gets the first array member / character:
print (f[f['digitStruct']['name'].value[0].item()].value[0][0])
print (f[f['digitStruct']['name'].value[0].item()].value[0][0].tostring().decode('ascii'))
#  this gets the entire array / filename:
print (f[f['digitStruct']['name'].value[0].item()][:])
print (f[f['digitStruct']['name'].value[0].item()][:].tostring().decode('ascii'))

每种方法最后2条打印语句的输出是相同的：

Output from last 2 print statements for each method is identical:

[[ 49]
 [ 46]
 [112]
 [110]
 [103]]
1.png

这篇关于在SVHN数据集中访问hdf5组的两种方式之间有什么区别？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！