Tensorflow 初始化给出了所有的 [英] Tensorflow initialization gives all ones
问题描述
张量流 1.12.0
在下面截取的代码中,似乎wrapped_rv_val 和seq_rv_val 应该是等价的,但实际上不是.相反,seq_rv_val 被正确地初始化为随机生成的 init_val 数组,但wrapped_rv_val 被设置为全1.这是怎么回事?
In the code snipped below, it seems that wrapped_rv_val and seq_rv_val should be equivalent, but they are not. Instead, seq_rv_val is correctly initialized to the randomly generated init_val array, but wrapped_rv_val is set to all ones. What's going on here?
import numpy as np
import tensorflow as tf
init_val = np.random.rand(1, 1, 16, 1).astype(np.dtype('float32'))
wrapped_rv = tf.nn.softmax(tf.get_variable('wrapped_rv', initializer=init_val))
var = tf.get_variable('seq_rv', initializer=init_val)
seq_rv = tf.nn.softmax(var, axis=2)
init_op = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init_op)
wrapped_rv_val = sess.run(wrapped_rv)
seq_rv_val = sess.run(seq_rv)
print("seq_rv_val: {0}".format(seq_rv_val.flatten()))
print("wrapped_rv_val: {0}".format(wrapped_rv_val.flatten()))
输出:
seq_rv_val: [0.28422353 0.12556878 0.18170598 0.19684952 0.21165217]
seq_rv_val: [0.28422353 0.12556878 0.18170598 0.19684952 0.21165217]
wrapped_rv_val: [1.1. 1. 1. 1.]
wrapped_rv_val: [1. 1. 1. 1. 1.]
推荐答案
事实上,seq_rv_val
和 wrapped_rv_val
都会正确初始化为随机生成的 init_val数组
,当您执行以下操作时.
In fact, seq_rv_val
and wrapped_rv_val
both will be correctly initialized to the randomly generated init_val array
when you do the following.
# change
wrapped_rv = tf.nn.softmax(tf.get_variable('wrapped_rv', initializer=init_val))
# to
wrapped_rv = tf.nn.softmax(tf.get_variable('wrapped_rv', initializer=init_val), axis=2)
接下来解释一下为什么wrapped_rv
被初始化为1,我们来看一下softmax
的公式.
Next I'll explain why wrapped_rv
is initialized to 1. Let's look at the formula of softmax
.
当您设置axis=2
时,分母求和项的数量将为16.但是当您设置axis=-1
(默认)时,分母求和项的数量将为1.所以分子和分母是一样的,当你把它设置为axis=-1
时,结果是1.你可以运行下面的例子来理解问题.
The number of denominator summation items will be 16 when you set axis=2
. But the number of denominator summation items will be 1 when you set axis=-1
(default). So the molecule is the same as the denominator and the result is 1 when you set it to axis=-1
.
You can run the following example to understand the problem.
import tensorflow as tf
y = tf.constant([[1],[0],[1]],dtype=tf.float32)
y1 = tf.constant([[1],[2],[3]],dtype=tf.float32)
y2 = tf.constant([[1],[3],[7]],dtype=tf.float32)
softmax_var1 = tf.nn.softmax(logits=y1)
softmax_var2 = tf.nn.softmax(logits=y2)
with tf.Session() as sess:
print(sess.run(softmax_var1))
print(sess.run(softmax_var2))
[[1.]
[1.]
[1.]]
[[1.]
[1.]
[1.]]
这篇关于Tensorflow 初始化给出了所有的的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!