火炬数据集的结构是什么？ [英] What is the structure of torch dataset?

问题描述

我开始使用火炬7，我想让我的数据集进行分类。我已经制作了像素图像和相应的标签。但是，我不知道如何将这些数据提供给火炬手。我从别人那里读了一些代码，发现他们正在使用扩展名为'.t7'的数据集，我认为它是一个张量类型。这样对吗？而且我想知道如何转换我的像素图像（实际上，我使用Matlab通过使用MNIST数据集）与t7扩展兼容的手电筒。必须有t7格式的数据集的结构，但是我找不到它（也用于标签）。总而言之，我有像素图像和标签，并希望将它们转换为兼容T7格式的手电筒。

提前感谢

解决方案

数据集'.t7'是标签的传感器的表。
例如以下lua代码：

  if（not paths.filep（cifar10torchsmall.zip））然后
 os.execute（'wget -c https://s3.amazonaws.com/torch7/data/cifar10torchsmall.zip'）
 os.execute（'unzip cifar10torchsmall.zip'）
 end 
 Readed_t7 = torch.load（'cifar10-train.t7'）
 print（Readed_t7）
  

将通过itorch返回：

  {
 data：ByteTensor  -  size：10000x3x32x32 
标签：ByteTensor  - 大小：10000 
} 
  

这意味着文件包含两个ByteTensor标记为数据，另一个标记为标签。

要回答您的问题，您应该首先阅读您的图像（使用torchx，例如： https://github.com/nicholas-leonard/torchx/blob/master/README .md ），然后将它们放在带有Tensor标签的表格中。以下代码只是一个草案来帮助您。它考虑的情况是：有两个类，所有的图像都在同一个文件夹中，并通过这些类排序。

 要求'torchx'; 
 
  - 读取所有数据集（所选扩展名为png）
文件= paths.indexdir（/ Path / to / your / images /，'png'，true）
 data1 = {} 
 for i = 1，files：size（）do 
 local img1 = image.load（files：filename（i），3）
 table.insert data1，img1）
 end 
 
  - 根据
 label1 = {} 
为i = 1创建标签表，＃data1 do 
如果i< = number_of_images_of_the_first_class然后
 label1 [i] = 1 
 else 
 label1 [i] = 2 
 end 
 end 
 
 - 调整表到Tensors 
 label = torch.Tensor（label1）
 data = torch.Tensor（＃data1,3,16,16）
 for i = 1，＃data1 do 
 data [i] = data1 [i] 
 end 
 
  - 创建表以保存
 Data_to_Write = {data = data，label = label} 
 
  - 在/ tmp 
 torch.save（/ tmp / Saved_Data.t7，Data_to_Write）中保存表
  

应该可以做一个不太可怕的代码，但这一个细节所有的步骤并且与火炬7和Jupyter 5.0.0一起使用。

希望它有帮助。

>

I am beginning to use torch 7 and I want to make my dataset for classification. I've already made pixel images and corresponding labels. However, I do not know how to feed those data to the torch. I read some codes from others and found out that they are using the dataset whose extension is '.t7' and I think it is a tensor type. Is it right? And I wonder how I can convert my pixel images(actually, I made them with Matlab by using MNIST dataset) into t7 extension compatible to the torch. There must be structure of dataset in the t7 format but I cannot find it (also for the labels too).

To sum up, I have pixel images and labels and want to convert those to t7 format compatible to the torch.

Thanks in advance!

解决方案

The datasets '.t7' are tables of labeled Tensors. For example the following lua code :

if (not paths.filep("cifar10torchsmall.zip")) then
    os.execute('wget -c https://s3.amazonaws.com/torch7/data/cifar10torchsmall.zip')
    os.execute('unzip cifar10torchsmall.zip')
end
Readed_t7 = torch.load('cifar10-train.t7')
print(Readed_t7)

Will return through itorch :

{
  data : ByteTensor - size: 10000x3x32x32
  label : ByteTensor - size: 10000
}

Which means the file contains a table of two ByteTensor one labeled "data" and the other one labeled "label".

To answer your question, you should first read your images (with torchx for example : https://github.com/nicholas-leonard/torchx/blob/master/README.md ) then put them in a table with your Tensor of label. The following code is just a draft to help you out. It considers the case where : there are two classes, all your images are in the same folder and are ordered through those classes.

require 'torchx';

--Read all your dataset (the chosen extension is png)
files = paths.indexdir("/Path/to/your/images/", 'png', true)
data1 = {}
for i=1,files:size() do
   local img1 = image.load(files:filename(i),3)
   table.insert(data1, img1)
end

--Create the table of label according to 
label1 = {}
for i=1, #data1 do
    if i <= number_of_images_of_the_first_class then
        label1[i] = 1
    else
        label1[i] = 2
    end
end

--Reshape the tables to Tensors
label = torch.Tensor(label1)
data = torch.Tensor(#data1,3,16,16)
for i=1, #data1 do
    data[i] = data1[i]
end

--Create the table to save
Data_to_Write = { data = data, label = label }

--Save the table in the /tmp
torch.save("/tmp/Saved_Data.t7", Data_to_Write)

It should be possible to make a less hideous code but this one details all the steps and works with torch 7 and Jupyter 5.0.0 .

Hope it helps.

Regards

这篇关于火炬数据集的结构是什么？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！