word2vec 如何从嵌入向量中给出一个热词向量? [英] How does word2vec give one hot word vector from the embedding vector?

问题描述

我了解 word2vec 的工作原理.

I understand how word2vec works.

我想使用 word2vec(skip-gram) 作为 RNN 的输入.输入是嵌入词向量.输出也是嵌入RNN生成的词向量.

I want to use word2vec(skip-gram) as input for RNN. Input is embedding word vector. Output is also embedding word vector generated by RNN.

问题来了！如何将输出向量转换为一个热词向量?我需要嵌入的逆矩阵，但我没有！

Here’s question! How can I convert the output vector to one hot word vector? I need inverse matrix of embeddings but I don’t have!

推荐答案

RNN 的输出不是嵌入.我们通过乘以适当的矩阵，将 RNN 单元中最后一层的输出转换为 vocabulary_size 的向量.

The output of an RNN is not an embedding. We convert the output from the last layer in an RNN cell into a vector of vocabulary_size by multiplying with an appropriate matrix.

看看PTB语言模型 示例以获得更好的想法.具体看133-136行:

Take a look at the PTB Language Model example to get a better idea. Specifically look at lines 133-136:

softmax_w = tf.get_variable("softmax_w", [size, vocab_size], dtype=data_type())
softmax_b = tf.get_variable("softmax_b", [vocab_size], dtype=data_type())
logits = tf.matmul(output, softmax_w) + softmax_b

上面的操作会给你logits.这个 logits 是你词汇量的概率分布.numpy.random.choice 可能会帮助您使用这些对数进行预测.

The above operation will give you logits. This logits is a probability distribution over your vocabulary. numpy.random.choice might help you to use these logits to make a prediction.

这篇关于word2vec 如何从嵌入向量中给出一个热词向量?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！