TensorFlow嵌入查询 [英] TensorFlow Embedding Lookup

问题描述

我正在尝试学习如何使用TensorFlow构建用于语音识别的RNN.首先，我想尝试一些在TensorFlow页面上放置的示例模型 TF-RNN

I am trying to learn how to build RNN for Speech Recognition using TensorFlow. As a start, I wanted to try out some example models put up on TensorFlow page TF-RNN

根据建议，我花了一些时间通过研究word2vec模型代码的基本版本来理解单词ID如何嵌入到密集表示(向量表示)中.我了解了tf.nn.embedding_lookup的实际作用，直到我实际上遇到了

As per what was advised, I had taken some time to understand how word IDs are embedded into a dense representation (Vector Representation) by working through the basic version of word2vec model code. I had an understanding of what tf.nn.embedding_lookup actually does, until I actually encountered the same function being used with two dimensional array in TF-RNN ptb_word_lm.py, when it did not make sense any more.

给定一个二维数组params和一个一维数组ids，函数tf.nn.embedding_lookup从参数中获取行，与ids中给定的索引相对应，并保留其输出维数正在返回.

Given a 2-d array params, and a 1-d array ids, function tf.nn.embedding_lookup fetches rows from params, corresponding to the indices given in ids, which holds with the dimension of output it is returning.

当尝试使用相同的参数和2-d数组ids时，tf.nn.embedding_lookup返回3-d数组，而不是我不明白为什么的2-d数组.

When tried with same params, and 2-d array ids, tf.nn.embedding_lookup returns 3-d array, instead of 2-d which I do not understand why.

我查阅了嵌入查询的手册，但是我仍然很难理解分区的工作原理以及返回的结果.我最近用tf.nn.embedding_lookup尝试了一个简单的示例，似乎每次都返回不同的值.这是由于分区涉及的随机性吗?

I looked up the manual for Embedding Lookup, but I still find it difficult to understand how the partitioning works, and the result that is returned. I recently tried some simple example with tf.nn.embedding_lookup and it appears that it returns different values each time. Is this behaviour due to the randomness involved in partitioning ?

请帮助我了解tf.nn.embedding_lookup的工作原理，以及为什么同时在word2vec_basic.py和ptb_word_lm.py中使用tf.nn.embedding_lookup的原因，即使用它们的目的是什么?

Please help me understand how tf.nn.embedding_lookup works, and why is used in both word2vec_basic.py, and ptb_word_lm.py i.e., what is the purpose of even using them ?

推荐答案

tf.nn.embedding_lookup

尝试使用相同的参数和2维数组ID时，tf.nn.embedding_lookup返回3维数组，而不是我不明白为什么的2维数组.

When tried with same params, and 2-d array ids, tf.nn.embedding_lookup returns 3-d array, instead of 2-d which I do not understand why.

当具有一维IDs [0, 1]列表时，该函数将返回嵌入列表[embedding_0, embedding_1]，其中embedding_0是形状为embedding_size的数组.例如，ID列表可能是一批单词.

When you had a 1-D list of ids [0, 1], the function would return a list of embeddings [embedding_0, embedding_1] where embedding_0 is an array of shape embedding_size. For instance the list of ids could be a batch of words.

现在，您有了ID的矩阵或ID列表的列表.例如，您现在有一批句子，即一批单词列表，即一系列单词列表.

Now, you have a matrix of ids, or a list of list of ids. For instance, you now have a batch of sentences, i.e. a batch of list of words, i.e. a list of list of words.

如果句子列表为:[[0, 1], [0, 3]](句子1为[0, 1]，句子2为[0, 3])，则该函数将计算嵌入矩阵，其形状为[2, 2, embedding_size]，看起来像:

If your list of sentences is: [[0, 1], [0, 3]] (sentence 1 is [0, 1], sentence 2 is [0, 3]), the function will compute a matrix of embeddings, which will be of shape [2, 2, embedding_size]and will look like:

[[embedding_0, embedding_1],
 [embedding_0, embedding_3]]

---

关于partition_strategy参数，您不必理会它.基本上，如果您在计算上有限制，则可以使用params而不是1个矩阵来给出嵌入矩阵的列表.

---

Concerning the partition_strategy argument, you don't have to bother about it. Basically, it allows you to give a list of embedding matrices as params instead of 1 matrix, if you have limitations in computation.

因此，您可以将形状为[1000, embedding_size]的嵌入矩阵拆分为十个形状为[100, embedding_size]的矩阵，并将此变量列表作为params传递. partition_strategy参数处理10个矩阵之间的词汇分布(1000个单词).

So, you could split your embedding matrix of shape [1000, embedding_size] in ten matrices of shape [100, embedding_size] and pass this list of Variables as params. The argument partition_strategy handles the distribution of the vocabulary (the 1000 words) among the 10 matrices.

这篇关于TensorFlow嵌入查询的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！