辍学训练 [英] Training with dropout
问题描述
由辍学导致的许多变薄层如何平均?在测试阶段要使用哪些砝码?我对此很困惑.因为每个变薄的层将学习不同的权重集.那么对每个精简网络分别进行反向传播吗?在这些精简网络之间权重是如何精确分配的?因为在测试时仅使用一个神经网络和一组权重.那么使用了哪一组权重?
How are the many thinned layers resulting from dropout averaged? And which weights are to be used during the testing stage? I'm really confused about this one. Because each thinned layers would learn a different set of weights. So backpropagation is done separately for each of the thinned networks? And how exactly are weights shared among these thinned networks? Because at testing time only one neural network is used and one set of weights. So which set of weights are used?
据说，每个训练案例都训练了一个不同的瘦化网络.培训案例到底是什么意思?您是说每个正向传播和反向传播都会训练一个不同的瘦化网络一次?然后，下一次正向和反向传播会训练另一个精简网络?如何学习体重?
It is said that a different thinned network is trained for each training case. What is exactly meant by training case? You mean each forward and backpropagation trains a different thinned network once? Then the next forward and backpropagation trains another thinned network? How are weights learned?
推荐答案
培训期间:
在退出"中，您只需要将该层的激活/输出的某些数量(退出概率)设为零即可.通常，会创建一个布尔掩码来删除这些激活.在进行反向传播时会使用这些遮罩.因此，将梯度应用于仅在前向道具中使用的权重.
In Dropout, you just force some number (dropout probability) of activations/outputs of that layer to be zero. Usually, a boolean mask is created to drop these activations. These masks are used while doing back propagation. So, gradients are applied to weights that are only used in forward prop.
测试时:
所有砝码都被使用.所有神经元都被保留(没有丢失)，但是该层的激活/输出按p(丢失概率)进行缩放，以标准化该层的整个输出.
All weights are used. All neurons are kept(no dropout), but the activations/outputs of that layer are scaled by p (dropout probability) for normalizing the whole output from that layer.
它只是一个网络，如上图所示(从此处使用:https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf )
Its just one network as shown in above figure (used from here : https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf )
问题:我不明白您所说的精简网络是什么意思.
Issues: I don't understand what do you mean by thinned networks.
我希望这会有所帮助.
这篇关于辍学训练的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
