在张量流训练中更新Keras BatchNormalization总体参数 [英] Keras BatchNormalization population parameters update while training in tensorflow

问题描述

我在带有Cuda 8.0和cuDNN 6的Ubuntu 16.04中将Keras 2.0.8和Tensorflow 1.3.0一起使用

我在模型中使用了两个BatchNormalization层( keras层)，并使用 tensorflow管道进行了训练.

我在这里面临两个问题-

即使训练 K.learning_phase 设置为 True，训练时
1. BatchNorm层人口参数(均值和方差)也不会更新 .结果，推论完全失败了.我需要一些有关如何在培训步骤之间手动更新这些参数的建议.
2. 第二，使用tensorflow 保存器 op 保存经过训练的模型后，当我尝试加载时，结果将无法转载.似乎权重正在改变.在保存负载操作中，有没有办法使权重保持相同?

解决方案

几周前，我遇到了同样的问题.在内部，keras层可以向模型添加其他更新操作(例如 解决方案

I ran into the same problem a few weeks ago. Internally, keras layers can add additional update operations to a model (e.g. batchnorm). So you need to run these additional ops explicitly. For the batchnorm these updates seem to be just some assign_ops which swap the current mean/variance with the new values. If you do not create a keras model this might work; assuming x is a tensor you like to normalize

bn = keras.layers.BatchNormalization()
x = bn(x)

....
sess.run([minimizer_op,bn.updates],K.learning_phase(): 1)

In my workflow, I am creating a keras model (w/o compiling it) and then run the following

model = keras.Model(inputs=inputs, outputs=prediction)
sess.run([minimizer_op,model.updates],K.learning_phase(): 1)

where inputs can be something like

inputs = [keras.layers.Input(tensor=input_variables)]

and outputs is a list of tensorflow tensors. The model seems to aggregate all additional updates operations between inputs and outputs automatically.

这篇关于在张量流训练中更新Keras BatchNormalization总体参数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！