批量标准化而不是输入标准化 [英] Batch normalization instead of input normalization

问题描述

我可以在输入层之后立即使用批处理归一化层，而不对数据进行归一化吗?我可以期望获得类似的效果/性能吗?

Can I use batch normalization layer right after input layer and not normalize my data? May I expect to get similar effect/performance?

在keras功能中，应该是这样的:

In keras functional it would be something like this:

x = Input (...)
x = Batchnorm(...)(x)
...

推荐答案

您可以做到.但是关于batchnorm的好处是，除了激活分布稳定之外，随着网络的学习，均值和std偏差很可能会迁移.

You can do it. But the nice thing about batchnorm, in addition to activation distribution stabilization, is that the mean and std deviation are likely migrate as the network learns.

有效地，在输入层之后立即设置batchnorm是数据预处理步骤.有时会很有帮助(例如线性回归).但是，一次计算整个训练样本的均值和方差要比逐批学习要容易和有效.请注意，就性能而言，batchnorm不是免费的，您不应滥用它.

Effectively, setting the batchnorm right after the input layer is a fancy data pre-processing step. It helps, sometimes a lot (e.g. in linear regression). But it's easier and more efficient to compute the mean and variance of the whole training sample once, than learn it per-batch. Note that batchnorm isn't free in terms of performance and you shouldn't abuse it.

这篇关于批量标准化而不是输入标准化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！