Tensorflow `tf.layers.batch_normalization` 不会向 `tf.GraphKeys.UPDATE_OPS` 添加更新操作 [英] Tensorflow `tf.layers.batch_normalization` doesn't add update ops to `tf.GraphKeys.UPDATE_OPS`
问题描述
以下代码(可复制/粘贴运行)说明了如何使用 tf.layers.batch_normalization
.
The following code (copy/paste runnable) illustrates using tf.layers.batch_normalization
.
import tensorflow as tf
bn = tf.layers.batch_normalization(tf.constant([0.0]))
print(tf.get_collection(tf.GraphKeys.UPDATE_OPS))
> [] # UPDATE_OPS collection is empty
使用 TF 1.5,文档(下面引用)明确指出在这种情况下 UPDATE_OPS 不应为空(https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization):
Using TF 1.5, the documentation (quoted below) clearly states that UPDATE_OPS should not be empty in this case (https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization):
注意:训练时,moving_mean和moving_variance需要为更新.默认情况下,更新操作位于tf.GraphKeys.UPDATE_OPS
,因此需要将它们作为依赖添加到train_op.例如:
Note: when training, the moving_mean and moving_variance need to be updated. By default the update ops are placed in
tf.GraphKeys.UPDATE_OPS
, so they need to be added as a dependency to the train_op. For example:
update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
with tf.control_dependencies(update_ops):
train_op = optimizer.minimize(loss)
推荐答案
只需将代码更改为训练模式(通过将 training
标志设置为 True
)作为引用中提到:
Just change your code to be in training mode (by setting the training
flag to True
) as mentioned in the quote:
注意:训练时,需要更新moving_mean和moving_variance.默认情况下,更新操作放置在 tf.GraphKeys.UPDATE_OPS 中,因此需要将它们作为依赖项添加到 train_op.
Note: when training, the moving_mean and moving_variance need to be updated. By default the update ops are placed in tf.GraphKeys.UPDATE_OPS, so they need to be added as a dependency to the train_op.
import tensorflow as tf
bn = tf.layers.batch_normalization(tf.constant([0.0]), training=True)
print(tf.get_collection(tf.GraphKeys.UPDATE_OPS))
将输出:
[< tf.Tensor 'batch_normalization/AssignMovingAvg:0' shape=(1,) dtype=float32_ref>,
< tf.Tensor 'batch_normalization/AssignMovingAvg_1:0' shape=(1,) dtype=float32_ref>]
Gamma 和 Beta 最终出现在 TRAINABLE_VARIABLES 集合中:
and Gamma and Beta end up in the TRAINABLE_VARIABLES collection:
print(tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES))
[<tf.Variable 'batch_normalization/gamma:0' shape=(1,) dtype=float32_ref>,
<tf.Variable 'batch_normalization/beta:0' shape=(1,) dtype=float32_ref>]
这篇关于Tensorflow `tf.layers.batch_normalization` 不会向 `tf.GraphKeys.UPDATE_OPS` 添加更新操作的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!