池化层或卷积层后的激活函数? [英] Activation function after pooling layer or convolutional layer?
问题描述
这些链接的理论表明，卷积网络的顺序是:Convolutional Layer - Non-linear Activation - Pooling Layer.
The theory from these links show that the order of Convolutional Network is: Convolutional Layer - Non-linear Activation - Pooling Layer.
但是，在这些网站的最后一个实现中，它说顺序是:卷积层 - 池化层 - 非线性激活
But, in the last implementation from those sites, it said that the order is: Convolutional Layer - Pooling Layer - Non-linear Activation
我也尝试过探索 Conv2D 操作语法，但没有激活函数，它只是与翻转内核的卷积.有人能帮我解释一下为什么会这样吗?
I've tried too to explore a Conv2D operation syntax, but there is no activation function, it's only convolution with flipped kernel. Can someone help me to explain why is this happen?
推荐答案
好吧，最大池化和单调递增的非线性通勤.这意味着 MaxPool(Relu(x)) = Relu(MaxPool(x)) 对于任何输入.所以在那种情况下结果是一样的.因此，技术上最好先通过最大池化进行子采样，然后再应用非线性(如果代价高昂，例如 sigmoid).在实践中，它通常以相反的方式完成 - 它的性能似乎没有太大变化.
Well, max-pooling and monotonely increasing non-linearities commute. This means that MaxPool(Relu(x)) = Relu(MaxPool(x)) for any input. So the result is the same in that case. So it is technically better to first subsample through max-pooling and then apply the non-linearity (if it is costly, such as the sigmoid). In practice it is often done the other way round - it doesn't seem to change much in performance.
至于 conv2D，它不翻转内核.它完全实现了卷积的定义.这是一个线性操作，因此您必须在下一步中自己添加非线性，例如theano.tensor.nnet.relu.
As for conv2D, it does not flip the kernel. It implements exactly the definition of convolution. This is a linear operation, so you have to add the non-linearity yourself in the next step, e.g. theano.tensor.nnet.relu.
这篇关于池化层或卷积层后的激活函数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
