如何解释生成对抗网络中的歧视者损失和生成者损失? [英] How to interpret the discriminator's loss and the generator's loss in Generative Adversarial Nets?

问题描述

我正在阅读人们对DCGAN的实现，尤其是在Tensorflow中的此.

在该实现中，作者绘制了鉴别器和生成器的损耗，如下所示(图像来自解决方案

不幸的是，就像您对GAN所说的那样，损失是非常不直观的.主要是由于生成器和鉴别器相互竞争这一事实，因此，一方面改进意味着另一方损失更高，直到另一方更好地了解接收到的损失，从而加剧了竞争对手的竞争等. >

现在应该经常发生的一件事(取决于您的数据和初始化)是鉴别器和生成器损耗都收敛到一些永久性数字，例如: (损失可以反弹一点是可以的-这只是模型试图自我完善的证据)

这种损失收敛通常表示GAN模型找到了一些最佳值，无法进一步提高，这也意味着它已经学到了足够的知识. (还请注意，数字本身通常并不能提供很多信息.)

以下是一些注意事项，希望对您有所帮助:

• 如果损失不能很好地收敛，这并不一定意味着该模型没有学到任何东西-检查所生成的示例，有时它们会变得足够好.或者，可以尝试更改学习率和其他参数.
• 如果模型能够很好地收敛，请仍然检查生成的示例-有时生成器会发现一个鉴别器无法将其与真实数据区分开的示例.麻烦的是它总是发出很少的东西，而不是创建任何新东西，这被称为模式崩溃.通常，为数据引入一些多样性会有所帮助.
• 由于香草GAN相当不稳定，我建议使用某些版本 DCGAN模型，因为它们包含卷积等功能 层和批处理规范化，应该有助于 收敛的稳定性. (上图是DCGAN而不是香草GAN的结果)
• 这是一些常识，但仍然存在:就像大多数神经网络结构调整模型一样，即更改其参数或/和体系结构以满足您的特定需求/数据可以改善模型或对其进行修正.

I am reading people's implementation of DCGAN, especially this one in tensorflow.

In that implementation, the author draws the losses of the discriminator and of the generator, which is shown below (images come from https://github.com/carpedm20/DCGAN-tensorflow):

Both the losses of the discriminator and of the generator don't seem to follow any pattern. Unlike general neural networks, whose loss decreases along with the increase of training iteration. How to interpret the loss when training GANs?

解决方案

Unfortunately, like you've said for GANs the losses are very non-intuitive. Mostly it happens down to the fact that generator and discriminator are competing against each other, hence improvement on the one means the higher loss on the other, until this other learns better on the received loss, which screws up its competitor, etc.

Now one thing that should happen often enough (depending on your data and initialisation) is that both discriminator and generator losses are converging to some permanent numbers, like this: (it's ok for loss to bounce around a bit - it's just the evidence of the model trying to improve itself)

This loss convergence would normally signify that the GAN model found some optimum, where it can't improve more, which also should mean that it has learned well enough. (Also note, that the numbers themselves usually aren't very informative.)

Here are a few side notes, that I hope would be of help:

• if loss haven't converged very well, it doesn't necessarily mean that the model hasn't learned anything - check the generated examples, sometimes they come out good enough. Alternatively, can try changing learning rate and other parameters.
• if the model converged well, still check the generated examples - sometimes the generator finds one/few examples that discriminator can't distinguish from the genuine data. The trouble is it always gives out these few, not creating anything new, this is called mode collapse. Usually introducing some diversity to your data helps.
• as vanilla GANs are rather unstable, I'd suggest to use some version of the DCGAN models, as they contain some features like convolutional layers and batch normalisation, that are supposed to help with the stability of the convergence. (the picture above is a result of the DCGAN rather than vanilla GAN)
• This is some common sense but still: like with most neural net structures tweaking the model, i.e. changing its parameters or/and architecture to fit your certain needs/data can improve the model or screw it.

这篇关于如何解释生成对抗网络中的歧视者损失和生成者损失?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！