带最大池的卷积神经网络(CNN) [英] Convolutional Neural Network (CNN) with max-pooling

问题描述

我已经用Python实现了一个简单的CNN程序，该程序可以在MNIST数据集上进行机器学习.我实现了3层:

I've implemented a simple CNN program with Python that can machine learn on the MNIST data set. I've implemented 3 layers:

1. ConvPoolLayer，它会卷积然后表示合并
2. FullyConnectedLayer，这是一个完全连接的隐藏层
3. SoftmaxLayer，基本上可以提供网络的softmax输出

我在ConvPoolLayer中实现了均值池.这行代码确实意味着在正向传播期间进行池化:

It's in the ConvPoolLayer that I've implemented mean pooling. Here's the line of code that does mean pooling during forward propagation:

# 'activation' is a numpy array of 3D activations from the convolutional code (not shown here)     
skimage.measure.block_reduce(activation, block_size=(1, 1, 2, 2), func=np.mean) 

这是等效的反向传播代码:

And here's the equivalent back-propagation code:

# delta is a numpy array of 3D error matrices back-propagated from the upper layers
delta = delta.repeat(2, axis=2).repeat(2, axis=3)

它所做的只是放大错误.

All it's doing is just upscaling the error.

我的问题是，如何在不损失性能的情况下实现反向传播以实现最大池化?或者，有没有更好的方法来执行此操作而无需调用函数?经过几次平均池化后，我的准确度达到了约90-95％，所以我想看看最大池化如何影响性能.

My question is, how do I implement the backpropagation for max pooling without loss in performance? Or, is there a better way to do this without a function call? I get around ~90-95% accuracy after a few iterations with mean pooling, so I'd like to see how max pooling affects performance.

如果有任何NumPy技巧可以在这里应用，我将很高兴学习它们.我想了解自己在CNN中会发生什么，为什么事情会以它们的方式工作，以及是否可以优化操作，所以使用框架不是我的选择.

If there are any NumPy tricks that can be applied here, I would be glad to learn them. I want to understand myself what happens in a CNN, why things work the way they do, and whether operations can be optimised, so using frameworks isn't an option for me.

感谢您的帮助！

推荐答案

1. [已更新]对于最大池使用正向传播:

1. [updated] For forward propagation for max pooling use :

skimage.measure.block_reduce(activation, block_size=(1, 1, 2, 2), func=np.max)

您对平均池的反向传播并不完全正确.您应根据合并单元的数量(在您的情况下为4)来指定增量.请参见幻灯片11上的等式，位于 http://www.slideshare.net/kuwajima/cnnbp

Your backpropagation for mean pooling is not fully correct. You should devide delta by number of pooled cells (4 in your case). See equation on slide 11 at http://www.slideshare.net/kuwajima/cnnbp

要传播最大池，您只需将增量分配给正向传递中具有最高值的像元即可.因此，在池化层的前向传递过程中，通常跟踪最大激活的索引(有时也称为交换器)，以便在反向传播期间有效地进行梯度路由.参见 http://cs231n.github.io/convolutional-networks/#pool

To propagate max pooling you need to assign delta only to cell with highest value in forward pass. Hence, during the forward pass of a pooling layer it is common to keep track of the index of the max activation (sometimes also called the switches) so that gradient routing is efficient during backpropagation. See http://cs231n.github.io/convolutional-networks/#pool

实施这种方法的效率很低:

Very inefficient way to implement this:

#forward
activationPrevious = np.copy(activation)
skimage.measure.block_reduce(activation, block_size=(1, 1, 2, 2), func=np.max)
maxs = activations.repeat(2, axis=2).repeat(2, axis=3)
mask = np.equal(activationPrevious, maxs).astype(int)

#backward
delta = delta.repeat(2, axis=2).repeat(2, axis=3)
delta = np.multiply(delta, mask)

这篇关于带最大池的卷积神经网络(CNN)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！