为什么自动编码器中的解码器在最后一层使用 sigmoid? [英] Why is the decoder in an autoencoder uses a sigmoid on the last layer?
问题描述
我正在查看这个工作变分自动编码器.
I am looking at this working variational auto encoder.
主类
class VAE(nn.Module):
def __init__(self):
super(VAE, self).__init__()
self.fc1 = nn.Linear(784, 400)
self.fc21 = nn.Linear(400, 20)
self.fc22 = nn.Linear(400, 20)
self.fc3 = nn.Linear(20, 400)
self.fc4 = nn.Linear(400, 784)
def encode(self, x):
h1 = F.relu(self.fc1(x))
return self.fc21(h1), self.fc22(h1)
def reparametrize(self, mu, logvar):
std = logvar.mul(0.5).exp_()
if torch.cuda.is_available():
eps = torch.cuda.FloatTensor(std.size()).normal_()
else:
eps = torch.FloatTensor(std.size()).normal_()
eps = Variable(eps)
return eps.mul(std).add_(mu)
def decode(self, z):
h3 = F.relu(self.fc3(z))
return F.sigmoid(self.fc4(h3))
def forward(self, x):
mu, logvar = self.encode(x)
z = self.reparametrize(mu, logvar)
return self.decode(z), mu, logvar
有
def decode(self, z):
h3 = F.relu(self.fc3(z))
return F.sigmoid(self.fc4(h3))
我无法向自己解释为什么最后一层在返回之前应该通过一个 sigmoid.
I can't explain to myself why the last layer should be passed through a sigmoid before returning.
请解释.
我只是在没有 sigmoid 的情况下进行了检查.结果还是不错的.现在我不确定是否需要它.
I just checked without the sigmoid. Results are still nice. Now I am not sure if it is needed or not.
推荐答案
正如 Jim J 在回答中提到的,sigmoid 强制输出范围为 [0, 1].在这种情况下,并不是因为我们想将输出解释为概率,而是强制将输出解释为灰度图像的像素强度.
As mentioned in the answer by Jim J, sigmoid forces the output to the range [0, 1]. In this case, it's not because we want to interpret the output as a probability, rather it's done to force the output to be interpreted as pixel intensity of a grey scale image.
如果你移除 sigmoid,神经网络将不得不知道所有的输出都应该在 [0, 1] 的范围内.sigmoid 可能有助于使学习过程更加稳定.
If you remove the sigmoid, the NN will have to learn that all the outputs should be in the range [0, 1]. The sigmoid might help making the learning process more stable.
这篇关于为什么自动编码器中的解码器在最后一层使用 sigmoid?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!