Pytorch Batchnorm层与Keras Batchnorm不同 [英] Pytorch Batchnorm layer different from Keras Batchnorm

问题描述

我试图将经过训练的BN权重从pytorch模型复制到等效的Keras模型中，但是我一直得到不同的输出.

I'm trying to copy pre-trained BN weights from a pytorch model to its equivalent Keras model but I keep getting different outputs.

我阅读了Keras和Pytorch BN文档，我认为区别在于它们计算均值"和"var"的方式.

I read Keras and Pytorch BN documentation and I think that the difference lies in the way they calculate the "mean" and "var".

火炬:

均值和标准差是在 迷你批次

The mean and standard-deviation are calculated per-dimension over the mini-batches

来源: Pytorch BatchNorm

因此，它们是样本的平均值.

Thus, they average over samples.

凯拉斯:

axis:整数，应归一化的轴(通常是 特征轴).例如，在Conv2D图层之后， data_format ="channels_first"，在BatchNormalization中设置axis = 1.

axis: Integer, the axis that should be normalized (typically the features axis). For instance, after a Conv2D layer with data_format="channels_first", set axis=1 in BatchNormalization.

来源: Keras BatchNorm

在这里，它们是功能(渠道)的平均值

and here they average over the features (channels)

正确的方法是什么?如何在模型之间传递BN权重?

What's the right way? How to transfer BN weights between the models?

推荐答案

您可以从pytorch模块的running_mean和running_var属性中检索moving_mean和moving_variance

you can retrieve moving_mean and moving_variance from running_mean and running_var attributes of pytorch module

# torch weights, bias, running_mean, running_var corresponds to keras gamma, beta, moving mean, moving average

weights = torch_module.weight.numpy()  
bias = torch_module.bias.numpy()  
running_mean =  torch_module.running_mean.numpy()
running_var =  torch_module.running_var.numpy()

keras_module.set_weights([weights, bias, running_mean, running_var])

这篇关于Pytorch Batchnorm层与Keras Batchnorm不同的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！