卷积层不匹配中的Keras维数 [英] Keras dimensionality in convolutional layer mismatch

问题描述

我正在尝试与Keras一起构建我的第一个神经网络.我的经验为零，我似乎无法弄清楚为什么我的尺寸不合适.我无法从他们的文档中弄清楚该错误是在抱怨什么，甚至是导致该错误的原因.

我的模型接受一个32字节的数字数组，并且应该在另一端提供布尔值.我想在输入字节数组上进行一维卷积.

arr1是32字节数组，arr2是布尔数组.

inputData = np.array(arr1)
inputData = np.expand_dims(inputData, axis = 2)

labelData = np.array(arr2)

print inputData.shape
print labelData.shape

model = k.models.Sequential()
model.add(k.layers.convolutional.Convolution1D(32,2, input_shape = (32, 1)))
model.add(k.layers.Activation('relu'))

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

model.add(k.layers.core.Dense(32))
model.add(k.layers.Activation('sigmoid'))

model.compile(loss = 'binary_crossentropy',
              optimizer = 'rmsprop',
              metrics=['accuracy'])
model.fit(
    inputData,labelData
)

打印的形状输出为 (1000，32，1)和(1000，)

我收到的错误是:

回溯(最近一次通话最后一次):文件"cnn/ init .py"，行 50英寸 inputData，labelData文件"/home/steve/Documents/cnn/env/local/lib/python2.7/site-packages/keras/models.py"， 适合的863号线 initial_epoch = initial_epoch)文件"/home/steve/Documents/cnn/env/local/lib/python2.7/site-packages/keras/engine/training.py"， 1358行，适合 batch_size = batch_size)文件"/home/steve/Documents/cnn/env/local/lib/python2.7/site-packages/keras/engine/training.py"， _standardize_user_data中的第1238行 exception_prefix ='target')文件"/home/steve/Documents/cnn/env/local/lib/python2.7/site-packages/keras/engine/training.py"， _standardize_input_data中的第128行 str(array.shape))ValueError:检查目标时发生错误:预期activation_5具有3个维，但数组的形状为(1000，1)

解决方案

好吧，在我看来，您需要更多地了解卷积网络:-)

您将在每个步骤中在您的序列上应用长度为2的32个过滤器.因此，如果我们遵循每一层之后的张量尺寸:

尺寸:(无，32、1)

model.add(k.layers.convolutional.Convolution1D(32,2, input_shape = (32, 1)))
model.add(k.layers.Activation('relu'))

尺寸:(无，31、32) (您的长度为2的过滤器遍历整个序列，因此该序列的长度现在为31)

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

尺寸:(无，30、32) (由于长度为2的过滤器，您又损失了一个值，但仍有32个)

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

尺寸:(无，29、32) (相同...)

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

尺寸:(无，28、32)

现在您想在其上使用一个Dense层...问题是Dense层将在您的3D输入上按如下方式工作:

model.add(k.layers.core.Dense(32))
model.add(k.layers.Activation('sigmoid'))

尺寸:(无，28、32)

这是您的输出.我觉得很奇怪的第一件事是，您希望从密集层中输出32个输出……您应该输入1而不是32.但是即使这样也无法解决您的问题.看看如果我们更改最后一层会发生什么:

model.add(k.layers.core.Dense(1))
model.add(k.layers.Activation('sigmoid'))

尺寸:(无，28、1)

之所以会这样，是因为您将密集层应用于"2D"张量.在将密集(1)层应用于输入[28，32]的情况下，它会产生形状为(32,1)的权重矩阵，并将其应用于28个向量，这样您就可以找到28个输出大小为1.

我建议解决此问题的方法是更改​​最后2层，如下所示:

model = k.models.Sequential()
model.add(k.layers.convolutional.Convolution1D(32,2, input_shape = (32, 1)))
model.add(k.layers.Activation('relu'))

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

# Only use one filter so that the output will be a sequence of 28 values, not a matrix.
model.add(k.layers.convolutional.Convolution1D(1,2))
model.add(k.layers.Activation('relu'))

# Change the shape from (None, 28, 1) to (None, 28)
model.add(k.layers.core.Flatten())

# Only one neuron as output to get the binary target.
model.add(k.layers.core.Dense(1))
model.add(k.layers.Activation('sigmoid'))

现在，最后两个步骤将从

中获取张量

(无，29，32)->(无，28，1)->(无，28)->(无，1)

希望对您有帮助.

ps.如果您想知道什么是无"，那是批次的尺寸，您一次不喂入1000个样品，而是分批喂入，并且由于值取决于所选内容，因此我们将其设为无".

再解释一下为什么序列长度在每一步都会丢失一个值.

假设您有一个由4个值组成的序列[x1 x2 x3 x4]，您想使用长度为2 [f1 f2]的过滤器对序列进行卷积.第一个值将由y1 = [f1 f2] * [x1 x2]给出，第二个值将是y2 = [f1 f2] * [x2 x3]，第三个值将是y3 = [f1 f2] * [x3 x4].然后，您到达了序列的末尾，无法继续.结果是序列[y1 y2 y3].

这是由于滤镜长度和序列边界处的效果所致.有多个选项，为了使输出长度完全相同，有些序列会用0填充.您可以使用参数'padding'选择该选项.您可以了解更多信息此处，并为此处的padding参数找到不同的可能的值.我鼓励您阅读最后一个链接，它提供了有关输入和输出形状的信息...

从文档中:

填充:有效"或相同"之一(不区分大小写). 有效"表示无填充". 相同"导致对输入进行填充，以使输出与原始输入具有相同的长度.

默认值为有效"，因此您无需填充示例.

我还建议您将keras版本升级到最新版本. Convolution1D现在是Conv1D，因此您可能会发现文档和教程令人困惑.

I'm trying to play around with Keras to build my first neural network. I have zero experience and I can't seem to figure out why my dimensionality isn't right. I can't figure it out from their docs what this error is complaining about, or even what layer is causing it.

My model takes in a 32byte array of numbers, and is supposed to give a boolean value on the other side. I want a 1D convolution on the input byte array.

arr1 is the 32byte array, arr2 is an array of booleans.

inputData = np.array(arr1)
inputData = np.expand_dims(inputData, axis = 2)

labelData = np.array(arr2)

print inputData.shape
print labelData.shape

model = k.models.Sequential()
model.add(k.layers.convolutional.Convolution1D(32,2, input_shape = (32, 1)))
model.add(k.layers.Activation('relu'))

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

model.add(k.layers.core.Dense(32))
model.add(k.layers.Activation('sigmoid'))

model.compile(loss = 'binary_crossentropy',
              optimizer = 'rmsprop',
              metrics=['accuracy'])
model.fit(
    inputData,labelData
)

The output of the print of shapes are (1000, 32, 1) and (1000,)

The error I receive is:

Traceback (most recent call last): File "cnn/init.py", line 50, in inputData,labelData File "/home/steve/Documents/cnn/env/local/lib/python2.7/site-packages/keras/models.py", line 863, in fit initial_epoch=initial_epoch) File "/home/steve/Documents/cnn/env/local/lib/python2.7/site-packages/keras/engine/training.py", line 1358, in fit batch_size=batch_size) File "/home/steve/Documents/cnn/env/local/lib/python2.7/site-packages/keras/engine/training.py", line 1238, in _standardize_user_data exception_prefix='target') File "/home/steve/Documents/cnn/env/local/lib/python2.7/site-packages/keras/engine/training.py", line 128, in _standardize_input_data str(array.shape)) ValueError: Error when checking target: expected activation_5 to have 3 dimensions, but got array with shape (1000, 1)

解决方案

Well It seems to me that you need to google a bit more about convolutional networks :-)

You are applying at each step 32 filters of length 2 over yout sequence. So if we follow the dimensions of the tensors after each layer :

Dimensions : (None, 32, 1)

model.add(k.layers.convolutional.Convolution1D(32,2, input_shape = (32, 1)))
model.add(k.layers.Activation('relu'))

Dimensions : (None, 31, 32) (your filter of length 2 goes over the whole sequence so the sequence is now of length 31)

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

Dimensions : (None, 30, 32) (you lose again one value because of your filters of length 2, but you still have 32 of them)

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

Dimensions : (None, 29, 32) (same...)

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

Dimensions : (None, 28, 32)

Now you want to use a Dense layer on top of that... the thing is that the Dense layer will work as follow on your 3D input :

model.add(k.layers.core.Dense(32))
model.add(k.layers.Activation('sigmoid'))

Dimensions : (None, 28, 32)

This is your output. First thing that I find weird is that you want 32 outputs out of your dense layer... You should have put 1 instead of 32. But even this will not fix your problem. See what happens if we change the last layer :

model.add(k.layers.core.Dense(1))
model.add(k.layers.Activation('sigmoid'))

Dimensions : (None, 28, 1)

This happens because you apply a dense layer to a '2D' tensor. What it does in case you apply a dense(1) layer to an input [28, 32] is that it produces a weight matrix of shape (32,1) that it applies to the 28 vectors so that you find yourself with 28 outputs of size 1.

What I propose to fix this is to change the last 2 layers like this :

model = k.models.Sequential()
model.add(k.layers.convolutional.Convolution1D(32,2, input_shape = (32, 1)))
model.add(k.layers.Activation('relu'))

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

model.add(k.layers.convolutional.Convolution1D(32,2))
model.add(k.layers.Activation('relu'))

# Only use one filter so that the output will be a sequence of 28 values, not a matrix.
model.add(k.layers.convolutional.Convolution1D(1,2))
model.add(k.layers.Activation('relu'))

# Change the shape from (None, 28, 1) to (None, 28)
model.add(k.layers.core.Flatten())

# Only one neuron as output to get the binary target.
model.add(k.layers.core.Dense(1))
model.add(k.layers.Activation('sigmoid'))

Now the last two steps will take your tensor from

(None, 29, 32) -> (None, 28, 1) -> (None, 28) -> (None, 1)

I hope this helps you.

ps. if you were wondering what None is , it's the dimension of the batch, you don't feed the 1000 samples at onces, you feed it batch by batch and as the value depends on what is chosen, by convension we put None.

EDIT :

Explaining a bit more why the sequences length loses one value at each step.

Say you have a sequence of 4 values [x1 x2 x3 x4], you want to use your filter of length 2 [f1 f2] to convolve over the sequence. The first value will be given by y1 = [f1 f2] * [x1 x2], the second will be y2 = [f1 f2] * [x2 x3], the third will be y3 = [f1 f2] * [x3 x4]. Then you reached the end of your sequence and cannot go further. You have as a result a sequnce [y1 y2 y3].

This is due to the filter length and the effects at the borders of your sequence. There are multiple options, some pad the sequence with 0's in order to get exactly the same length of output... You can chose that option with the parameter 'padding'. You can read more about this here and find the different values possible for the padding argument here. I encourage you to read this last link, it gives informations about input and output shapes...

From the doc :

padding: One of "valid" or "same" (case-insensitive). "valid" means "no padding". "same" results in padding the input such that the output has the same length as the original input.

the default is 'valid', so you don't pad in your example.

I also recommend you to upgrade your keras version to the latest. Convolution1D is now Conv1D, so you might find the doc and tutorials confusing.

这篇关于卷积层不匹配中的Keras维数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！