带有嵌入层的 Keras LSTM 自动编码器 [英] Keras LSTM autoencoder with embedding layer

问题描述

我正在尝试在 Keras 中构建文本 LSTM 自动编码器.我想使用嵌入层，但我不确定如何实现.代码如下所示.

I am trying to build a text LSTM autoencoder in Keras. I want to use an embedding layer but I'am not sure how to implement this. The code looks like this.

inputs = Input(shape=(timesteps, input_dim))
embedding_layer = Embedding(numfeats + 1,
                            EMBEDDING_DIM,
                            weights=[data_gen.get_embedding_matrix()],
                            input_length=maxlen,
                            trainable=False)

embedded_sequence = embedding_layer(inputs)
encoded = LSTM(num_units)(inputs)

decoded = RepeatVector(timesteps)(encoded)
decoded = LSTM(???, return_sequences=True)(decoded)

sequence_autoencoder = Model(inputs, decoded)

sequence_autoencoder.compile(loss='binary_crossentropy', optimizer='adam')

我不确定如何将输出解码为目标序列(显然是输入序列).

I am not sure how to decode the output into the target sequence (which is obviously the input sequence).

推荐答案

没有办法在解码器中实现反向嵌入层，因为嵌入层是不可微的.可能还有其他方法可以解决:

There is no way to implement an inversed embedding layer in the decoder, because the embedding layer is not differentiable. There are probably other ways to work around:

1. 从嵌入层的输出构造自动编码器，到具有相似维度的层.然后使用最近邻或其他算法从那里生成词序列.

1. construct the autoencoder from the output of the embedding layer, to a layer with a similar dimension. Then use the nearest neighbor or other algorithms to generate the word sequence from there.

构建一个非对称自编码器，使用时间分布层和密集层来降低 LSTM 输出的维度.

construct an asymmetric autoencoder, using the time distributed layer and dense layers to reduce the dimension of LSTM output.

希望这会有所帮助.

这篇关于带有嵌入层的 Keras LSTM 自动编码器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！