Keras 嵌入层掩码.为什么 input_dim 需要是 |vocabulary|+ 2? [英] Keras embedding layer masking. Why does input_dim need to be |vocabulary| + 2?

问题描述

在 Embedding 的 Keras 文档中 https://keras.io/layers/embeddings/，对mask_zero给出的解释是

In the Keras docs for Embedding https://keras.io/layers/embeddings/, the explanation given for mask_zero is

mask_zero:输入值 0 是否是应屏蔽的特殊填充"值.这在使用可能需要可变长度输入的循环层时很有用.如果为 True，则模型中的所有后续层都需要支持屏蔽，否则将引发异常.如果 mask_zero 设置为 True，则索引 0 不能在词汇表中使用(input_dim 应等于 |vocabulary| + 2).

mask_zero: Whether or not the input value 0 is a special "padding" value that should be masked out. This is useful when using recurrent layers which may take variable length input. If this is True then all subsequent layers in the model need to support masking or an exception will be raised. If mask_zero is set to True, as a consequence, index 0 cannot be used in the vocabulary (input_dim should equal |vocabulary| + 2).

为什么 input_dim 需要是 2 + 词汇中的单词数?假设0被屏蔽不能使用，不应该只是1+字数吗?另一个额外的条目是什么?

Why does input_dim need to be 2 + number of words in vocabulary? Assuming 0 is masked and can't be used, shouldn't it just be 1 + number of words? What is the other extra entry for?

推荐答案

我认为那里的文档有点误导.在正常情况下，您将 n 输入数据索引 [0, 1, 2, ..., n-1] 映射到向量，因此您的 input_dim 应该与您拥有的元素一样多

I believe the docs are a bit misleading there. In the normal case you are mapping your n input data indices [0, 1, 2, ..., n-1] to vectors, so your input_dim should be as many elements as you have

input_dim = len(vocabulary_indices)

一种等效的(但有点令人困惑)的表达方式，以及文档的方式，就是说

An equivalent (but slightly confusing) way to say this, and the way the docs do, is to say

1 + 输入数据中出现的最大整数索引.

1 + maximum integer index occurring in the input data.

input_dim = max(vocabulary_indices) + 1

如果您启用屏蔽，值 0 将被区别对待，因此您将 n 索引增加 1:[0, 1, 2, ..., n-1, n]，因此你需要

If you enable masking, value 0 is treated differently, so you increment your n indices by one: [0, 1, 2, ..., n-1, n], thus you need

input_dim = len(vocabulary_indices) + 1

或者替代

input_dim = max(vocabulary_indices) + 2

文档在这里变得特别混乱

The docs become especially confusing here as they say

(input_dim 应该等于 |vocabulary| + 2)

(input_dim should equal |vocabulary| + 2)

我将 |x| 解释为集合的基数(相当于 len(x))，但作者似乎是指

where I would interpret |x| as the cardinality of a set (equivalent to len(x)), but the authors seem to mean

2 + 输入数据中出现的最大整数索引.

2 + maximum integer index occurring in the input data.

这篇关于Keras 嵌入层掩码.为什么 input_dim 需要是 |vocabulary|+ 2?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！