Theano中卷积神经网络的无监督预训练 [英] Unsupervised pre-training for convolutional neural network in theano

问题描述

我想设计一个深层网络，在其顶部具有一个(或多个)卷积层(CNN)和一个或多个完全连接的隐藏层.
对于具有完全连接层的深度网络，theano中提供了无监督预训练的方法，例如，使用去噪自动编码器或 RBM .

I would like to design a deep net with one (or more) convolutional layers (CNN) and one or more fully connected hidden layers on top.
For deep network with fully connected layers there are methods in theano for unsupervised pre-training, e.g., using denoising auto-encoders or RBMs.

我的问题是:我如何在卷积层中实现无人值守的预训练阶段(在theano中)?

My question is: How can I implement (in theano) an unsupervised pre-training stage for convolutional layers?

我不希望得到完整的实施方案，但是希望获得指向优秀教程或可靠参考的链接.

I do not expect a full implementation as an answer, but I would appreciate a link to a good tutorial or a reliable reference.

推荐答案

本文描述了一种用于构建堆栈式卷积自动编码器的方法.基于该论文和一些Google搜索，我得以实现上述网络.基本上，您需要的所有内容都在Theano卷积网络和去噪自动编码器教程中进行了描述，但有一个关键例外:如何逆转卷积网络中的最大池化步骤.我能够使用此讨论-最棘手的部分是确定W_prime的正确尺寸，因为这将取决于前馈滤波器的大小和池化比率.这是我的反相函数:

This paper describes an approach for building a stacked convolutional autoencoder. Based on that paper and some Google searches I was able to implement the described network. Basically, everything you need is described in the Theano convolutional network and denoising autoencoder tutorials with one crucial exception: how to reverse the max-pooling step in the convolutional network. I was able to work that out using a method from this discussion - the trickiest part is figuring out the right dimensions for W_prime as these will depend on the feed forward filter sizes and the pooling ratio. Here is my inverting function:

    def get_reconstructed_input(self, hidden):
        """ Computes the reconstructed input given the values of the hidden layer """
        repeated_conv = conv.conv2d(input = hidden, filters = self.W_prime, border_mode='full')

        multiple_conv_out = [repeated_conv.flatten()] * np.prod(self.poolsize)

        stacked_conv_neibs = T.stack(*multiple_conv_out).T

        stretch_unpooling_out = neibs2images(stacked_conv_neibs, self.pl, self.x.shape)

        rectified_linear_activation = lambda x: T.maximum(0.0, x)
        return rectified_linear_activation(stretch_unpooling_out + self.b_prime.dimshuffle('x', 0, 'x', 'x'))

这篇关于Theano中卷积神经网络的无监督预训练的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！