当我将is_training = False设置为训练时间时，为什么Tensorflow BN层中的Moving_mean和Moving _variance会变为nan? [英] Why would moving_mean and moving _variance in Tensorflow BN layer become nan when I set is_training=False in training time?

问题描述

在训练期间，我想保持BN层不变，所以我将is_training = False传递给:

In the training time, I want to keep BN layer unchange, so I pass is_training=False to：

tf.contrib.layers.batch_norm(tensor_go_next, decay=0.9, center=True, scale=True, epsilon=1e-9,
                                              updates_collections=tf.GraphKeys.UPDATE_OPS,
                                              is_training=False, scope=name_bn_scope)

，并且没有将name_bn_scope/gamma:0 name_bn_scope/beta:0放入培训var_list.

and didn't put name_bn_scope/gamma:0 name_bn_scope/beta:0 to train var_list.

经过训练后，伽玛和贝塔值仍然相同，这正是我想要的.但是，经过训练后，moving_mean和moving_variance将成为nan矩阵，从而使精度达到0.1％.

After training, gamma and beta are still the same, which is what I want exactly. But the moving_mean and moving _variance would become nan matrix after training, which lead to the 0.1% accuracy.

我不明白为什么，不是is_taining =虚假的力张量流来保持Moving_mean和Moving_variance不变吗?我该如何解决和实施呢?

I don't understand why, dosen't is_taining=False force tensorflow to keep moving_mean and moving _variance unchanged? How can I fix and implement this?

BN层折磨了我很长时间， 请帮帮我！

BN layer has tortured me for a so long time, Please help me！

推荐答案

哈哈，我明白了:应该注释如下所示的代码块！ train_op跑了.由于我不想训练他们，所以应该将其删除.)

Aha, I figure it out: the code block as shown below should be commented!(which is used to force Tensorflow to chagne moving_mean/moving_variance in bn layer when train_op ran. Since I don't want to chagne them in training, then it should be removed.)

update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
with tf.control_dependencies(update_ops):
    train_op = optimizer.minimize(loss, name = 'train_op', var_list = var_list_to_train)

我还了解到，当被困在bug中时，也许出门休息一下是找出如何定位bug并加以解决的最佳方法，这有点像深入学习中的技巧局部最小值.

I also learned that when trapped in bugs, maybe go outside to take a break is the best way to figure out how to locate bugs and then solve it, which is a little bit like tricks in deep-learning to get out from local minimum.

这篇关于当我将is_training = False设置为训练时间时，为什么Tensorflow BN层中的Moving_mean和Moving _variance会变为nan?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！