PyTorch LSTM - 使用词嵌入代替 nn.Embedding() [英] PyTorch LSTM - using word embeddings instead of nn.Embedding()

问题描述

nn.Embedding() 对于学习 LSTM 是必不可少的吗?

Is the nn.Embedding() essential for learning for an LSTM?

我在 PyTorch 中使用 LSTM 来预测 NER - 类似任务的示例在这里 - https://pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html

I am using an LSTM in PyTorch to predict NER - example of a similar task is here - https://pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html

代码方面，我使用的代码几乎与上面教程中的代码相同.

Code wise, I am using code almost identical to the code in the tutorial above.

唯一的细节是——我使用 word2Vec 而不是 nn.Embedding().

The only detail is - I am using word2Vec instead of nn.Embedding().

所以我移除了 nn.Embedding() 层并直接从 word2Vec 中提供前向函数的特征.RNN 不会学习.

So I remove the nn.Embedding() layer and provide the forward function the features from the word2Vec directly. The RNN does not learn.

因此，nn.Embedding() 对于学习 LSTM 是否必不可少?

Hence, Is the nn.Embedding() essential for learning for an LSTM?

推荐答案

nn.Embedding 为您提供了一个嵌入层.

nn.Embedding provides an embedding layer for you.

这意味着该层会获取您的词标记 ID 并将其转换为词向量.

This means that the layer takes your word token ids and converts these to word vectors.

您可以在训练过程中学习 nn.Embedding 层的权重，或者您也可以加载预训练的嵌入权重.

You can learn the weights for your nn.Embedding layer during the training process, or you can alternatively load pre-trained embedding weights.

当您想使用预训练的 word2vec(嵌入)模型时，您只需将预训练的权重加载到 nn.Embedding 层.

When you want to use a pre-trained word2vec (embedding) model, you just load the pre-trained weights into the nn.Embedding layer.

您可以在 此处查看如何使用 gensim 库加载 word2vec 嵌入层.

我希望这会有所帮助.

这篇关于PyTorch LSTM - 使用词嵌入代替 nn.Embedding()的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！