张量流基本lstm_cell中的trainable_weights和trainable_variables有什么区别? [英] What is the difference between the trainable_weights and trainable_variables in the tensorflow basic lstm_cell?

查看:240
本文介绍了张量流基本lstm_cell中的trainable_weights和trainable_variables有什么区别?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

当尝试使用记录的在此,我偶然发现了trainable_weights和trainable_variables属性.

源代码还没有真可悲的是,对于像我这样的菜鸟来说,这确实很有用.一点点的实验确实产生了以下信息:两者的布局完全相同,是长度为2的列表,其中第一个条目为tf.形状的变量为(2 * num_units,4 * num_units),列表的第二个条目的形状为(4 * num_units,),其中num_units是初始化BasicLSTMCell的num_units.现在,对我而言,直观的猜测是,第一个列表项是lstm的四个内部层的权重的串联,第二个项是各个偏差的串联,显然适合这些偏差的预期大小.

现在的问题是,两者之间实际上是否有区别?我认为它们可能仅仅是从rnn_cell类继承这些结果的结果吗?

解决方案

RNNCell 继承的 Layer 类的源代码中:

  @propertydef trainable_variables(自己):返回 self.trainable_weights 

请参见此处.RNN类似乎并没有覆盖该定义-我会假设它存在于具有可训练变量的特殊图层类型中,这些变量并不完全符合权重"的要求.批量标准化将浮现在脑海,但是不幸的是,在我的源代码中,我找不到任何提及 trainable_variables 的内容( GraphKeys.TRAINABLE_VARIABLES 除外).>

While trying to copy the weights of a LSTM Cell in Tensorflow using the Basic LSTM Cell as documented here, i stumbled upon both the trainable_weights and trainable_variables property.

Source code has not really been informative for a noob like me sadly. A little bit of experimenting did yield the following information though: Both have the exact same layout, being a list of length two, where the first entry is a tf.Variable of shape: (2*num_units, 4*num_units), the second entry of the list is of shape (4*num_units,), where num_units is the num_units from initializing the BasicLSTMCell. The intuitive guess for me is now, that the first list item is a concatenation of the weights of the four internal layers of the lstm, the second item being a concatenation of the respective biases, fitting the expected sizes of these obviously.

Now the question is, whether there is actually any difference between these? I assume they might just be a result of inheriting these from the rnn_cell class?

解决方案

From the source code of the Layer class that RNNCell inherits from:

@property
def trainable_variables(self):
    return self.trainable_weights

See here. The RNN classes don't seem to overwrite this definition -- I would assume it's there for special layer types that have trainable variables that don't quite qualify as "weights". Batch normalization would come to mind, but unfortunately I can't find any mention of trainable_variables in that one's source code (except for GraphKeys.TRAINABLE_VARIABLES which is different).

这篇关于张量流基本lstm_cell中的trainable_weights和trainable_variables有什么区别?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
相关文章
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆