合并层或卷积层之后的激活功能? [英] Activation function after pooling layer or convolutional layer?

问题描述

这些链接的理论表明，卷积网络的顺序为:Convolutional Layer - Non-linear Activation - Pooling Layer.

The theory from these links show that the order of Convolutional Network is: Convolutional Layer - Non-linear Activation - Pooling Layer.

1. 神经网络和深度学习(方程(125)
2. 深度学习书(第304页，第一段)
3. Lenet(等式)
4. 此标题中的来源

1. Neural networks and deep learning (equation (125)
2. Deep learning book (page 304, 1st paragraph)
3. Lenet (the equation)
4. The source in this headline

但是，在这些站点的最后一个实现中，它表示顺序为:Convolutional Layer - Pooling Layer - Non-linear Activation

But, in the last implementation from those sites, it said that the order is: Convolutional Layer - Pooling Layer - Non-linear Activation

1. network3.py
2. 源代码LeNetConvPoolLayer类

1. network3.py
2. The sourcecode, LeNetConvPoolLayer class

我也尝试过探索Conv2D操作语法，但是没有激活函数，它只是与翻转内核的卷积.有人可以帮我解释为什么会这样吗?

I've tried too to explore a Conv2D operation syntax, but there is no activation function, it's only convolution with flipped kernel. Can someone help me to explain why is this happen?

推荐答案

嗯，最大池化和单调增加的非线性通勤能力.这意味着对于任何输入，MaxPool(Relu(x))= Relu(MaxPool(x)).因此，在这种情况下，结果是相同的.因此，从技术上来说，最好先通过最大池化对子样本进行采样，然后再应用非线性函数(如果代价高昂，例如S型).在实践中，通常是用另一种方式完成的-它的性能似乎并没有太大变化.

Well, max-pooling and monotonely increasing non-linearities commute. This means that MaxPool(Relu(x)) = Relu(MaxPool(x)) for any input. So the result is the same in that case. So it is technically better to first subsample through max-pooling and then apply the non-linearity (if it is costly, such as the sigmoid). In practice it is often done the other way round - it doesn't seem to change much in performance.

对于conv2D，它不翻转内核.它精确地实现了卷积的定义.这是线性运算，因此您必须在下一步中自行添加非线性，例如theano.tensor.nnet.relu.

As for conv2D, it does not flip the kernel. It implements exactly the definition of convolution. This is a linear operation, so you have to add the non-linearity yourself in the next step, e.g. theano.tensor.nnet.relu.

这篇关于合并层或卷积层之后的激活功能?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！