回归caffe的测试标签，不允许浮动? [英] Test labels for regression caffe, float not allowed?

问题描述

我在用caffe做回归，我的test.txt和train.txt文件是这样的:

I am doing regression using caffe, and my test.txt and train.txt files are like this:

/home/foo/caffe/data/finetune/flickr/3860781056.jpg 2.0  
/home/foo/caffe/data/finetune/flickr/4559004485.jpg 3.6  
/home/foo/caffe/data/finetune/flickr/3208038920.jpg 3.2  
/home/foo/caffe/data/finetune/flickr/6170430622.jpg 4.0  
/home/foo/caffe/data/finetune/flickr/7508671542.jpg 2.7272

我的问题是当我在阅读时使用浮动标签时，caffe 似乎不允许像 2.0 这样的浮动标签，例如 'test.txt' 文件仅 caffe识别

My problem is it seems caffe does not allow float labels like 2.0, when I use float labels while reading, for example the 'test.txt' file caffe only recognizes

共 1 张图片

这是错误的.

但是，例如，当我将文件中的 2.0 更改为 2 并且以下几行相同时，caffe 现在给出

But when I for example change the 2.0 to 2 in the file and the following lines same, caffe now gives

共 2 张图片

暗示浮动标签是造成问题的原因.

implying that the float labels are responsible for the problem.

任何人都可以在这里帮我解决这个问题，我肯定需要使用浮点标签进行回归，所以有人知道解决方法或解决方案吗?提前致谢.

Can anyone help me here, to solve this problem, I definitely need to use float labels for regression, so does anyone know about a work around or solution for this? Thanks in advance.

编辑对于面临类似问题的任何人使用 caffe 用 CSV 数据训练 Lenet 可能会有所帮助.感谢@Shai.

EDIT For anyone facing a similar issue use caffe to train Lenet with CSV data might be of help. Thanks to @Shai.

推荐答案

当使用图像数据集输入层(带有 lmdb 或 leveldb 后端)时，caffe 只支持一个每个输入图像的整数标签.

When using the image dataset input layer (with either lmdb or leveldb backend) caffe only supports one integer label per input image.

如果你想做回归，并使用浮点标签，你应该尝试使用 HDF5 数据层.参见例如这个问题.

If you want to do regression, and use floating point labels, you should try and use the HDF5 data layer. See for example this question.

在python中你可以使用h5py包来创建hdf5文件.

In python you can use h5py package to create hdf5 files.

import h5py, os
import caffe
import numpy as np

SIZE = 224 # fixed size to all images
with open( 'train.txt', 'r' ) as T :
    lines = T.readlines()
# If you do not have enough memory split data into
# multiple batches and generate multiple separate h5 files
X = np.zeros( (len(lines), 3, SIZE, SIZE), dtype='f4' ) 
y = np.zeros( (len(lines),1), dtype='f4' )
for i,l in enumerate(lines):
    sp = l.split(' ')
    img = caffe.io.load_image( sp[0] )
    img = caffe.io.resize( img, (SIZE, SIZE, 3) ) # resize to fixed size
    # you may apply other input transformations here...
    # Note that the transformation should take img from size-by-size-by-3 and transpose it to 3-by-size-by-size
    # for example
    # transposed_img = img.transpose((2,0,1))[::-1,:,:] # RGB->BGR
    X[i] = transposed_img
    y[i] = float(sp[1])
with h5py.File('train.h5','w') as H:
    H.create_dataset( 'X', data=X ) # note the name X given to the dataset!
    H.create_dataset( 'y', data=y ) # note the name y given to the dataset!
with open('train_h5_list.txt','w') as L:
    L.write( 'train.h5' ) # list all h5 files you are going to use

一旦您拥有所有 h5 文件和列出它们的相应测试文件，您就可以将 HDF5 输入层添加到您的 train_val.prototxt:

Once you have all h5 files and the corresponding test files listing them you can add an HDF5 input layer to your train_val.prototxt:

 layer {
   type: "HDF5Data"
   top: "X" # same name as given in create_dataset!
   top: "y"
   hdf5_data_param {
     source: "train_h5_list.txt" # do not give the h5 files directly, but the list.
     batch_size: 32
   }
   include { phase:TRAIN }
 }

<小时>

说明:
当我说caffe 只支持每个输入图像一个整数标签"时，我并不是说 leveldb/lmdb 容器是有限的，我指的是 caffe 的工具，特别是 convert_imageset 工具.
仔细观察，似乎 caffe 将 Datum 类型的数据存储在 leveldb/lmdb 中，并且这种类型的label"属性被定义为整数(请参阅 caffe.proto) 因此当使用 caffe 接口到 leveldb/lmdb 时，您被限制为每个图像一个 int32 标签.

---

Clarification:
When I say "caffe only supports one integer label per input image" I do not mean that the leveldb/lmdb containers are limited, I meant the tools of caffe, specifically the convert_imageset tool.
At closer inspection, it seems like caffe stores data of type Datum in leveldb/lmdb and the "label" property of this type is defined as integer (see caffe.proto) thus when using caffe interface to leveldb/lmdb you are restricted to a single int32 label per image.

这篇关于回归caffe的测试标签，不允许浮动?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！