初始化 LSTM 隐藏状态 Tensorflow/Keras [英] Initializing LSTM hidden state Tensorflow/Keras

问题描述

有人可以解释我如何在张量流中初始化 LSTM 的隐藏状态吗?我正在尝试构建 LSTM 循环自动编码器，因此在训练该模型后，我想将学习到的无监督模型的隐藏状态转移到有监督模型的隐藏状态.使用当前的 API 甚至可能吗?这是我正在尝试重新创建的论文:

Can someone explain how can I initialize hidden state of LSTM in tensorflow? I am trying to build LSTM recurrent auto-encoder, so after i have that model trained i want to transfer learned hidden state of unsupervised model to hidden state of supervised model. Is that even possible with current API? This is paper I am trying to recreate:

http://papers.nips.cc/paper/5949-半监督序列学习.pdf

推荐答案

是的 - 这是可能的，但确实很麻烦.我们来看一个例子.

Yes - this is possible but truly cumbersome. Let's go through an example.

1. 定义模型:

1. Defining a model:

from keras.layers import LSTM, Input
from keras.models import Model

input = Input(batch_shape=(32, 10, 1))
lstm_layer = LSTM(10, stateful=True)(input)

model = Model(input, lstm_layer)
model.compile(optimizer="adam", loss="mse")

首先构建和编译模型很重要，因为在编译时初始状态会被重置.此外 - 您需要指定一个 batch_shape 其中 batch_size 被指定，因为在这种情况下我们的网络应该是 stateful(这是通过设置 batch_sizecode>stateful=True 模式.

It's important to build and compile model first as in compilation the initial states are reset. Moreover - you need to specify a batch_shape where batch_size is specified as in this scenario our network should be stateful (which is done by setting a stateful=True mode.

现在我们可以设置初始状态的值:

Now we could set the values of initial states:

import numpy
import keras.backend as K

hidden_states = K.variable(value=numpy.random.normal(size=(32, 10)))
cell_states = K.variable(value=numpy.random.normal(size=(32, 10)))

model.layers[1].states[0] = hidden_states
model.layers[1].states[1] = cell_states 

请注意，您需要将状态作为 keras 变量提供.states[0] 保存隐藏状态，states[1] 保存细胞状态.

Note that you need to provide states as a keras variables. states[0] holds hidden states and states[1] holds cell states.

希望有所帮助.

这篇关于初始化 LSTM 隐藏状态 Tensorflow/Keras的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！