权重以什么顺序保存在Tensorflow中的LSTM内核中 [英] In what order are weights saved in a LSTM kernel in Tensorflow

问题描述

我查看了Tensorflow中LSTMCell的已保存权重. 它具有一个大内核和偏权重.

I looked into the saved weights for a LSTMCell in Tensorflow. It has one big kernel and bias weights.

内核的尺寸为

(input_size + hidden_size)*(hidden_size*4)

现在，据我了解，这是封装4个输入到隐藏层仿射变换以及4个隐藏到隐藏层变换.

Now from what I understand this is encapsulating 4 input to hidden layer affine transforms as well as 4 hidden to hidden layer transforms.

所以应该有4个大小的矩阵

So there should be 4 matrices of size

input_size*hidden_size

大小为4

hidden_size*hidden_size

有人可以告诉我还是将我指向TF保存这些代码的代码，这样我就可以将内核矩阵分解为较小的矩阵.

Can someone tell me or point me to the code where TF saves these, so I can break the kernel matrix into smaller matrices.

推荐答案

权重如另一个答案中所述进行合并，但顺序为: 其中c是上下文，而h是历史记录.

The weights are combined as mentioned in the other answer, but the order is: where c is the context and h is the history.

input_c,      input_h
new_input_c,  new_input_h
forget_c,     forget_h
output_c,     output_h

相关代码在此处

if self._state_is_tuple:
  c, h = state
else:
  c, h = array_ops.split(value=state, num_or_size_splits=2, axis=one)

gate_inputs = math_ops.matmul(
    array_ops.concat([inputs, h], 1), self._kernel)
gate_inputs = nn_ops.bias_add(gate_inputs, self._bias)

# i = input_gate, j = new_input, f = forget_gate, o = output_gate
i, j, f, o = array_ops.split(
    value=gate_inputs, num_or_size_splits=4, axis=one)

这篇关于权重以什么顺序保存在Tensorflow中的LSTM内核中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！