如何在 Tensorflow 2.0 中通过 Xavier 规则进行权重初始化? [英] How to do weight initialization by Xavier rule in Tensorflow 2.0?

问题描述

TF 2.0 碰巧去掉了 contrib 库.因此，像 tf.contrib.conv2d 或 tf.contrib.layers.variance_scaling_initializer 这样的好东西都消失了.也就是说，您认为在不使用 Keras(或使用一些 numpy hack 进行初始化)的情况下，在 TF2.0 中进行 Xavier 初始化的最佳方法是什么?

TF 2.0 happened to get rid of contrib library. Therefore, all the goodies like tf.contrib.conv2d or tf.contrib.layers.variance_scaling_initializer are gone. That said, what do you think would be the best way to do Xavier initialization in TF2.0 without using Keras (or initializing with some numpy hack)?

也就是说，我坚持使用 tf.nn.conv2d 并且对于该功能，我是提供权重的人:

Namely, I am sticking to tf.nn.conv2d and for that function I am the one providing the weights:

filters = tf.Variable(tf.zeros([3, 3, 32, 64]))
??? # something should happen hear, i guess
net = tf.nn.conv2d(input, filters)

注意:以防万一您使用的是 TF 的第一个版本，您可以使用:

Note: Just in case you are using the first version of TF you can just go with:

filters = tf.get_variable("w", shape=[3,3, 32, 64],
           initializer=tf.contrib.layers.xavier_initializer()) 
# no tf.contrib in 2.0, no tf.get_variable in 2.0

推荐答案

在 tensorflow 2.0 中，您有一个 tf.initializer 包，其中包含您需要的所有类似 Keras 的初始化程序.

In tensorflow 2.0 you have a package tf.initializer with all the Keras-like initializers you need.

Xavier 初始值设定项与 Glorot Uniform 初始值设定项相同.因此，要创建一个 (3,3) 变量，其中的值从该初始化程序中采样，您只需:

The Xavier initializer is the same as the Glorot Uniform initializer. Thus, to create a (3,3) variable with values sampled from that initializer you can just:

shape = (3,3)
initializer = tf.initializers.GlorotUniform()
var = tf.Variable(initializer(shape=shape))

这篇关于如何在 Tensorflow 2.0 中通过 Xavier 规则进行权重初始化?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！