具有嵌入层的Keras LSTM自动编码器 [英] Keras LSTM autoencoder with embedding layer

问题描述

我正在尝试在Keras中构建文本LSTM自动编码器.我想使用嵌入层，但不确定如何实现.代码看起来像这样.

I am trying to build a text LSTM autoencoder in Keras. I want to use an embedding layer but I'am not sure how to implement this. The code looks like this.

inputs = Input(shape=(timesteps, input_dim))
embedding_layer = Embedding(numfeats + 1,
                            EMBEDDING_DIM,
                            weights=[data_gen.get_embedding_matrix()],
                            input_length=maxlen,
                            trainable=False)

embedded_sequence = embedding_layer(inputs)
encoded = LSTM(num_units)(inputs)

decoded = RepeatVector(timesteps)(encoded)
decoded = LSTM(???, return_sequences=True)(decoded)

sequence_autoencoder = Model(inputs, decoded)

sequence_autoencoder.compile(loss='binary_crossentropy', optimizer='adam')

我不确定如何将输出解码为目标序列(显然是输入序列).

I am not sure how to decode the output into the target sequence (which is obviously the input sequence).

推荐答案

由于嵌入层不可区分，因此无法在解码器中实现反向嵌入层.可能还有其他解决方法:

There is no way to implement an inversed embedding layer in the decoder, because the embedding layer is not differentiable. There are probably other ways to work around:

1. 将自动编码器从嵌入层的输出构造为具有相似尺寸的层.然后使用最近的邻居或其他算法从那里生成单词序列.

1. construct the autoencoder from the output of the embedding layer, to a layer with a similar dimension. Then use the nearest neighbor or other algorithms to generate the word sequence from there.

构造一个不对称的自动编码器，使用时间分布层和密集层来减小LSTM输出的尺寸.

construct an asymmetric autoencoder, using the time distributed layer and dense layers to reduce the dimension of LSTM output.

希望这会有所帮助.

这篇关于具有嵌入层的Keras LSTM自动编码器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！