如何在 TensorFlow 上进行 Xavier 初始化 [英] How to do Xavier initialization on TensorFlow

问题描述

我正在将我的 Caffe 网络移植到 TensorFlow，但它似乎没有 xavier 初始化.我正在使用 truncated_normal 但这似乎使训练变得更加困难.

I'm porting my Caffe network over to TensorFlow but it doesn't seem to have xavier initialization. I'm using truncated_normal but this seems to be making it a lot harder to train.

推荐答案

在 Tensorflow 2.0 以及 tf.contrib.* 和 tf.get_variable() 已弃用.为了进行 Xavier 初始化，您现在必须切换到:

In Tensorflow 2.0 and further both tf.contrib.* and tf.get_variable() are deprecated. In order to do Xavier initialization you now have to switch to:

init = tf.initializers.GlorotUniform()
var = tf.Variable(init(shape=shape))
# or a oneliner with a little confusing brackets
var = tf.Variable(tf.initializers.GlorotUniform()(shape=shape))

Glorot uniform 和 Xavier uniform 是同一个初始化类型的两个不同名称.如果您想了解更多有关如何在 TF2.0 中使用或不使用 Keras 初始化的信息，请参阅 文档.

Glorot uniform and Xavier uniform are two different names of the same initialization type. If you want to know more about how to use initializations in TF2.0 with or without Keras refer to documentation.

这篇关于如何在 TensorFlow 上进行 Xavier 初始化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！