深度信念网络与卷积神经网络 [英] Deep Belief Networks vs Convolutional Neural Networks

问题描述

我是神经网络领域的新手，我想知道深度信念网络和卷积网络之间的区别。
此外，是否有一个深度卷积网络，是深信念和卷积神经网络的组合？

这是我收集到现在。

对于图片分类问题，深度信任网络有许多图层，每个图层都使用贪婪的层次策略。
例如，如果我的图片大小为50 x 50，并且我想要一个4层的深层网络，即

2. 隐藏图层1（HL1）


3. 隐藏图层2（HL2）


4. 输出图层

我的输入图层将有50 x 50 = 2500个神经元，HL1 = 1000个神经元（说），HL2 = 100个神经元= 10神经元，
为了训练输入层和HL1之间的权重（W1），我使用自动编码器（2500-1000-2500）并学习大小为2500×1000的W1（这是无监督的学习）。然后，我通过第一隐藏层前馈所有图像以获得一组特征，然后使用另一自动编码器（1000-100-1000）获得下一组特征，并最终使用softmax层（100-10）进行分类。 （只学习最后一层的权重（HL2 - 输出是softmax层）是监督学习）。

（我可以使用RBM代替自动编码器） / p>

如果使用卷积神经网络解决了相同的问题，那么对于50x50输入图像，我将开发一个仅使用7 x 7说）。我的图层为

1. 输入图层（7 x 7 = 49个神经元）


2. HL1


3. 输出层（Softmax）


ol>



并且为了学习权重，我从大小50 x 50的图像中取7 x 7的补丁，并通过卷积层前馈，因此我将有25个不同的特征地图





然后我使用一个说11x11的窗口用于合并手，因此得到25作为池层的输出的大小（4×4）的特征图。我使用这些特征地图进行分类。





在学习权重时，我不使用深层思维网络（无监督学习）监督学习和同时学习所有层的权重。这是正确的，还是有任何其他方式来学习权重？





我是否理解正确？





所以，如果我想使用DBN的图像分类，我应该将所有的图像调整到一个特定的大小（比如说200x200），并在输入图层有许多神经元，而在CNN的情况下，我只训练更小的输入（例如对于大小为200×200的图像为10×10）并且将学习的权重卷积到整个图像上。





DBN提供比CNN更好的结果还是纯粹依赖于数据集？





谢谢。





卷积神经网络在目前关于基准计算机视觉数据集（例如MNIST）的文献中表现优于DBN。如果数据集不是计算机视觉的，那么DBNs可以表现得更好。在理论上，DBN应该是最好的模型，但是很难准确地估计联合概率。您可能感兴趣的Lee et。 al（2009）研究了卷积深信念网络，它们将两者结合起来。




I am new to the field of neural networks and I would like to know the difference between Deep Belief Networks and Convolutional Networks. Also, is there a Deep Convolutional Network which is the combination of Deep Belief and Convolutional Neural Nets?

This is what I have gathered till now. Please correct me if I am wrong.

For an image classification problem, Deep Belief networks have many layers, each of which is trained using a greedy layer-wise strategy. For example, if my image size is 50 x 50, and I want a Deep Network with 4 layers namely

1. Input Layer
2. Hidden Layer 1 (HL1)
3. Hidden Layer 2 (HL2)
4. Output Layer

My input layer will have 50 x 50 = 2500 neurons, HL1 = 1000 neurons (say) , HL2 = 100 neurons (say) and output layer = 10 neurons, in order to train the weights (W1) between Input Layer and HL1, I use an AutoEncoder (2500 - 1000 - 2500) and learn W1 of size 2500 x 1000 (This is unsupervised learning). Then I feed forward all images through the first hidden layers to obtain a set of features and then use another autoencoder ( 1000 - 100 - 1000) to get the next set of features and finally use a softmax layer (100 - 10) for classification. (only learning the weights of the last layer (HL2 - Output which is the softmax layer) is supervised learning).

(I could use RBM instead of autoencoder).

If the same problem was solved using Convolutional Neural Networks, then for 50x50 input images, I would develop a network using only 7 x 7 patches (say). My layers would be

1. Input Layer (7 x 7 = 49 neurons)
2. HL1 (25 neurons for 25 different features) - (convolution layer)
3. Pooling Layer
4. Output Layer (Softmax)

And for learning the weights, I take 7 x 7 patches from images of size 50 x 50, and feed forward through convolutional layer, so I will have 25 different feature maps each of size (50 - 7 + 1) x (50 - 7 + 1) = 44 x 44.

I then use a window of say 11x11 for pooling hand hence get 25 feature maps of size (4 x 4) for as the output of the pooling layer. I use these feature maps for classification.

While learning the weights, I don't use the layer wise strategy as in Deep Belief Networks (Unsupervised Learning), but instead use supervised learning and learn the weights of all the layers simultaneously. Is this correct or is there any other way to learn the weights?

Is what I have understood correct?

So if I want to use DBN's for image classification, I should resize all my images to a particular size (say 200x200) and have that many neurons in the input layer, whereas in case of CNN's, I train only on a smaller patch of the input (say 10 x 10 for an image of size 200x200) and convolve the learnt weights over the entire image?

Do DBNs provide better results than CNNs or is it purely dependent on the dataset?

Thank You.

解决方案

Generally speaking, DBNs are generative neural networks that stack Restricted Boltzmann Machines (RBMs) . You can think of RBMs as being generative autoencoders; if you want a deep belief net you should be stacking RBMs and not plain autoencoders as Hinton and his student Yeh proved that stacking RBMs results in sigmoid belief nets.

Convolutional neural networks have performed better than DBNs by themselves in current literature on benchmark computer vision datasets such as MNIST. If the dataset is not a computer vision one, then DBNs can most definitely perform better. In theory, DBNs should be the best models but it is very hard to estimate joint probabilities accurately at the moment. You may be interested in Lee et. al's (2009) work on Convolutional Deep Belief Networks which looks to combine the two.

这篇关于深度信念网络与卷积神经网络的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！