tf.contrib.layers.embed_sequence() 是干什么用的? [英] tf.contrib.layers.embed_sequence() is for what?

问题描述

我在最新的 Tensorflow 示例中找到了 tf.contrib.layers.embed_sequence() 函数，但它并未包含在主 API 中.我不知道为什么.任何有关其工作原理的解释将不胜感激.

I found tf.contrib.layers.embed_sequence() function in the lastest Tensorflow examples, but it is not included in the main API . I don't know why. Any explanation about how it works would be appreciated.

推荐答案

我能想到 tensorflow.contrib.layers.embed_sequence 有用的两个主要原因:

I can think of two main reasons why tensorflow.contrib.layers.embed_sequence is useful:

1. 在构建具有多个以特征为输入的门的神经网络模型时，通过使用tensorflow.contrib.layers.embed_sequence，您可以在保留深度的同时减少网络中的参数数量.例如，它消除了 LSTM 的每个门都执行自己的特征线性投影的需要.
2. 它允许任意输入形状，这有助于实现简单灵活.

1. When building a neural network model that has multiple gates that take features as input, by using tensorflow.contrib.layers.embed_sequence, you can reduce the number of parameters in your network while preserving depth. For example, it eliminates the need for each gates of the LSTM to perform its own linear projection of features.
2. It allows for arbitrary input shapes, which helps the implementation be simple and flexible.

假设我有一个看起来像这样的数据集:

Let us say that I have a data set which looks something like this:

[("城市垃圾堆","垃圾"),("城市被车辆堵塞","交通")]

我想取每个元组的第一个元素，它是一个单词序列.这些词需要以向量形式嵌入.作为第一步，它们应该被转换为索引或数字.例如，在这种情况下，词汇将是:

I want to take the first element of each tuple which is a sequence of words. The words need to be embedded in a vector form. As the first step, they should be converted as indices or numbers. For example, in this case, the vocabulary will be:

vocab = [{'garbage':1},
         {'piles':2},
         {'in':3},
         {'the':4},
         {'city':5},
         {'is':6},
         {'clogged':7},
         {'with':8},
         {'vehicles':9}]

编码后的文本如下所示:

The encoded text will look like this:

features = [[1, 2, 3, 4, 5], [5, 6, 7, 8, 9]]

您将此编码文本作为 features 批量传递给此函数:

You pass this encoded text as features to this function in batches:

features_embedded = tf.contrib.layers.embed_sequence(
    ids=features,
    vocab_size=len(vocab),
    embed_dim=EMBEDDING_SIZE,
    scope='words'
)

现在，使用索引(1 到 5)表示的每个单词都嵌入到大小为 EMBEDDING_SIZE 的向量中.

Now, every word which is represented using the indices (1 to 5), becomes embedded into a vector of size EMBEDDING_SIZE.

如果批大小为 2(即一批中有 2 个序列)且 EMBEDDING_SIZE 为 10，则输出将是形状为 (2, 5, 10)

If the batch size is 2 (ie. 2 sequences in one batch) and EMBEDDING_SIZE is 10, the output will be a matrix of shape (2, 5, 10)

示例输出:

[[[0.1, 0.3, 0.4, 0.2, 0.5, 0.2, 0.2, 0.2, 0.4, 0.1], # garbage
  [0.1, 0.3, 0.4, 0.2, 0.5, 0.2, 0.1, 0.2, 0.4, 0.1], # piles
  [0.1, 0.3, 0.4, 0.2, 0.5, 0.2, 0.4, 0.2, 0.4, 0.1], # in
  [0.1, 0.3, 0.4, 0.2, 0.5, 0.3, 0.1, 0.2, 0.4, 0.1], # the
  [0.1, 0.3, 0.4, 0.2, 0.5, 0.2, 0.1, 0.2, 0.4, 0.6]], # city
 [sent2]]

sent2 的编码方式类似(5 x 10 矩阵).

sent2 is encoded similarly ( 5 x 10 matrix).

希望这很清楚.

这篇关于tf.contrib.layers.embed_sequence() 是干什么用的?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！