在python中访问SVHN数据集中的数据 [英] Accessing data in SVHN dataset in python

问题描述

我试图从包含digitStruct.mat文件的tar.gz文件中提取数据. 我使用了以下代码片段:

I tried to extract data from tar.gz file which contains digitStruct.mat file. I used the following code snippet:

train_dataset = h5py.File('./train/digitStruct.mat')

我想从该对象本身访问bbox和名称详细信息. 例如:

I want to access the bbox and name details from this object itself. for eg:

train_dataset[0]

应输出如下内容:

{'boxes': [{'height': 219.0,
'label': 1.0,
'left': 246.0,
'top': 77.0,
'width': 81.0},
{'height': 219.0, 'label': 9.0, 'left': 323.0, 'top': 81.0, 'width': 96.0}],
 'filename': '1.png'}

我搜索了它，并在此链接上找到了一些帮助:

I searched for it and found the some help on this link:

h5py，在SVHN中访问数据集中的数据

但是上面的链接涉及创建单独的函数get_box_data(index，hdf5_data)和get_name(index，hdf5_data)来检索相应索引的值. 但是，我想直接从变量名train_dataset [index]中访问它.

But the above link involves creating seperate functions get_box_data(index, hdf5_data) and get_name(index, hdf5_data) to retrieve value for the corresponding index. However, I want access it directly from the variable name train_dataset[index].

推荐答案

好的，我想我发现了我在上面的评论中提到的内容.它将.mat v7.3格式的SVHN HDF5文件转换为更易于使用的文件.文件名输入为dsFileName=. (我只有6个要转换的测试文件，因此没有添加输入机制.)它需要一个名为yourfilename.mat的文件并将其转换为yourfilename.h5.第二个文件更容易使用(更小，更快！).新的.h5文件有一个名为digitStruct的数据集，每行上都有以下记录:

Ok, I think I found what I mentioned in the comments above. It converts a SVHN HDF5 file in .mat v7.3 format into something simpler to work with. The file name is entered as dsFileName=. (I only had 6 test files to convert, so didn't add an input mechanism.) It takes a file named: yourfilename.mat and converts to yourfilename.h5. The second file is much easier to work with (and smaller and faster!). The new .h5 file has a dataset named digitStruct with the following records on each line:

• 名称:字符串(文件名，例如1.png)
• 标签:具有数字值(0-9)的字符串
• 左:图像边界框左
• 顶部:图像边界框顶部
• 宽度:图像边框的宽度
• 高度:图像边框的高度

• name: String (file name, eg 1.png)
• label: String with digit value (0-9)
• left: image bounding box left
• top: image bounding box top
• width: image bounding box width
• height: image bounding box height

注意:这将调用在github上共享的代码. URL和归因包含在下面的源代码中.

Note: This calls code shared on github. URL and Attribution included in the source code below.

import h5py
import numpy as np
import os
import digitStruct
## Note digitStruct.py source found at:
## https://github.com/prijip/Py-Gsvhn-DigitStruct-Reader/blob/master/digitStruct.py

# Main
if __name__ == "__main__":

    dsFileName = 'Stanford/extra/digitStruct.mat'
    print ('Working on',os.path.split(dsFileName))

    print ('Create .h5 called',os.path.splitext(dsFileName)[0]+'.h5')
    h5f = h5py.File(os.path.splitext(dsFileName)[0]+'.h5', 'w')
    print ('Created',os.path.split(h5f.filename))

# Count number of images in digitStruct.mat file [/name] dataset
    mat_f = h5py.File(dsFileName)
    num_img = mat_f['/digitStruct/name'].size
    mat_f.close()

    ds_dtype = np.dtype ( [('name','S16'), ('label','S10'), ('left','f8'),
                            ('top','f8'), ('width','f8'), ('height','f8')] )
    ds_recarray = np.recarray ( (10,) , dtype=ds_dtype )
    ds_table = h5f.create_dataset('digitStruct', (2*num_img,), dtype=ds_dtype, maxshape=(None,) )

    idx_dtype = np.dtype ( [('name','S16'), ('first','i4'), ('length','i4')] )
##    idx_recarray = np.recarray ( (1,) , dtype=idx_dtype )
    idx_table = h5f.create_dataset('idx_digitStruct', (num_img,), dtype=idx_dtype, maxshape=(None,) )

    imgCounter = 0
    lblCounter = 0

    for dsObj in digitStruct.yieldNextDigitStruct(dsFileName):
        if (imgCounter % 1000 == 0) :
               print(dsObj.name)

        if (idx_table.shape[0] < imgCounter ) : # resize idx_table as needed
            idx_table.resize(idx_table.shape[0]+1000, axis=0)

        idx_table[imgCounter,'name'] = dsObj.name
        idx_table[imgCounter,'first'] = lblCounter
        idx_table[imgCounter,'length'] = len(dsObj.bboxList)

        raCounter = 0

        for bbox in dsObj.bboxList:

            ds_recarray[raCounter]['name'] = dsObj.name
            ds_recarray[raCounter]['label'] = bbox.label
            ds_recarray[raCounter]['left'] = bbox.left
            ds_recarray[raCounter]['top'] = bbox.top
            ds_recarray[raCounter]['width'] = bbox.width
            ds_recarray[raCounter]['height'] = bbox.height
            raCounter += 1
            lblCounter += 1

        if (ds_table.shape[0] < lblCounter ) :   # resize ds_table as needed
            ds_table.resize(ds_table.shape[0]+1000, axis=0)
        ds_table[lblCounter-raCounter:lblCounter] = ds_recarray[0:raCounter]

        imgCounter += 1

##        if imgCounter >= 2000:
##            break

    print ('Total images processed:', imgCounter )
    print ('Total labels processed:', lblCounter )

    ds_table.resize(lblCounter, axis=0)
    idx_table.resize(imgCounter, axis=0)

    h5f.close()

这篇关于在python中访问SVHN数据集中的数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！