Caffe中的预测-例外：输入Blob参数与网络输入不匹配 [英] Prediction in Caffe - Exception: Input blob arguments do not match net inputs
问题描述
我正在使用Caffe使用非常简单的CNN结构对非图像数据进行分类。我在使用尺寸为nx 1 x 156 x 12的HDF5-数据上训练网络时没有问题。但是，我在分类新数据时遇到了困难。
I'm using Caffe for classifying non-image data using a quite simple CNN structure. I've had no problems training my network on my HDF5-data with dimensions n x 1 x 156 x 12. However, I'm having difficulties classifying new data.
如何我做一个没有任何预处理的简单正向传递？我的数据已经标准化，并且具有适用于Caffe的正确尺寸（已经用于训练网络）。下面是我的代码和CNN结构。
How do I do a simple forward pass without any preprocessing? My data has been normalized and have correct dimensions for Caffe (it's already been used to train the net). Below is my code and the CNN structure.
编辑：我已将问题隔离到pycaffe.py中的函数'_Net_forward'和发现问题出在self.input字典为空。谁能解释为什么？该集合应该等于来自新测试数据的集合：
I've isolated the problem to the function '_Net_forward' in pycaffe.py and found that the issue arises as the self.input dict is empty. Can anyone explain why that is? The set is supposed to be equal to the set coming from the new test data:
if set(kwargs.keys()) != set(self.inputs):
raise Exception('Input blob arguments do not match net inputs.')
我现在使用IO方法将数据转换为基准时，我的代码发生了一些变化（请参见下文）。这样，我就用正确的数据填充了kwargs变量。
My code has changed a bit as I now use the IO methods for converting the data into datum (see below). In that way I've filled the kwargs variable with the correct data.
即使是很小的提示也将不胜感激！
Even small hints would be greatly appreciated!
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
# Make sure that caffe is on the python path:
caffe_root = '' # this file is expected to be run from {caffe_root}
import sys
sys.path.insert(0, caffe_root + 'python')
import caffe
import os
import subprocess
import h5py
import shutil
import tempfile
import sklearn
import sklearn.datasets
import sklearn.linear_model
import skimage.io
def LoadFromHDF5(dataset='test_reduced.h5', path='Bjarke/hdf5_classification/data/'):
f = h5py.File(path + dataset, 'r')
dat = f['data'][:]
f.close()
return dat;
def runModelPython():
model_file = 'Bjarke/hdf5_classification/conv_v2_simple.prototxt'
pretrained = 'Bjarke/hdf5_classification/data/train_iter_10000.caffemodel'
test_data = LoadFromHDF5()
net = caffe.Net(model_file, pretrained)
caffe.set_mode_cpu()
caffe.set_phase_test()
user = test_data[0,:,:,:]
datum = caffe.io.array_to_datum(user.astype(np.uint8))
user_dat = caffe.io.datum_to_array(datum)
user_dat = user_dat.astype(np.uint8)
out = net.forward_all(data=np.asarray([user_dat]))
if __name__ == '__main__':
runModelPython()
CNN原型
name: "CDR-CNN"
layers {
name: "data"
type: HDF5_DATA
top: "data"
top: "label"
hdf5_data_param {
source: "Bjarke/hdf5_classification/data/train.txt"
batch_size: 10
}
include: { phase: TRAIN }
}
layers {
name: "data"
type: HDF5_DATA
top: "data"
top: "label"
hdf5_data_param {
source: "Bjarke/hdf5_classification/data/test.txt"
batch_size: 10
}
include: { phase: TEST }
}
layers {
name: "feature_conv"
type: CONVOLUTION
bottom: "data"
top: "feature_conv"
blobs_lr: 1
blobs_lr: 2
convolution_param {
num_output: 10
kernel_w: 12
kernel_h: 1
stride_w: 1
stride_h: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
}
}
}
layers {
name: "conv1"
type: CONVOLUTION
bottom: "feature_conv"
top: "conv1"
blobs_lr: 1
blobs_lr: 2
convolution_param {
num_output: 14
kernel_w: 1
kernel_h: 4
stride_w: 1
stride_h: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
}
}
}
layers {
name: "pool1"
type: POOLING
bottom: "conv1"
top: "pool1"
pooling_param {
pool: MAX
kernel_w: 1
kernel_h: 3
stride_w: 1
stride_h: 3
}
}
layers {
name: "conv2"
type: CONVOLUTION
bottom: "pool1"
top: "conv2"
blobs_lr: 1
blobs_lr: 2
convolution_param {
num_output: 120
kernel_w: 1
kernel_h: 5
stride_w: 1
stride_h: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
}
}
}
layers {
name: "fc1"
type: INNER_PRODUCT
bottom: "conv2"
top: "fc1"
blobs_lr: 1
blobs_lr: 2
weight_decay: 1
weight_decay: 0
inner_product_param {
num_output: 84
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layers {
name: "accuracy"
type: ACCURACY
bottom: "fc1"
bottom: "label"
top: "accuracy"
include: { phase: TEST }
}
layers {
name: "loss"
type: SOFTMAX_LOSS
bottom: "fc1"
bottom: "label"
top: "loss"
}
推荐答案
这里是我在Caffe Google网上论坛上得到的Evan Shelhamer的答案：
self ._inputs确实适用于手动输入或部署输入，如
所定义。一个原型。要通过pycaffe在
中运行带有数据层的网络，只需调用不带参数的net.forward（）即可。不需要
来更改火车或测试网的定义。
self._inputsis indeed for the manual or "deploy" inputs as defined by the input fields in a prototxt. To run a net with data layers in through pycaffe, just callnet.forward()without arguments. No need to change the definition of your train or test nets.
例如，请参见Python LeNet示例。
事实上，我认为 Caffe教程，单元格6：
In fact I think it's clearer in the Instant Recognition with Caffe tutorial, cell 6:
# Feed in the image (with some preprocessing) and classify with a forward pass.
net.blobs['data'].data[...] = transformer.preprocess('data', caffe.io.load_image(caffe_root + 'examples/images/cat.jpg'))
out = net.forward()
print("Predicted class is #{}.".format(out['prob'].argmax()))
换句话说，要使用pycaffe生成预测的输出及其概率，一旦训练了模型，就必须首先用您的输入，然后使用 net.forward（）执行前向传递。
In other words, to generate the predicted outputs as well as their probabilities using pycaffe, once you have trained your model, you have to first feed the data layer with your input, then perform a forward pass with net.forward().
或者，如其他答案所指出的，您可以使用与用于定义受训网络的部署原型相似的deploy原型，但是删除输入和输出层，并在开头添加以下内容（显然是根据您的输入维度）：
Alternatively, as pointed out in other answers, you can use a deploy prototxt that is similar to the one you use to define the trained network but removing the input and output layers, and add the following at the beginning (obviously adapting according to your input dimension):
name: "your_net"
input: "data"
input_dim: 1
input_dim: 1
input_dim: 1
input_dim: 250
这就是他们的在 CIFAR10教程中使用。
That's what they use in the CIFAR10 tutorial.
（真的应该更好地记录下pycaffe…）
这篇关于Caffe中的预测-例外：输入Blob参数与网络输入不匹配的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
