Keras嵌入层蒙版。为什么input_dim需要| vocalbulary | + 2？ [英] Keras embedding layer masking. Why does input_dim need to be |vocabulary| + 2?

问题描述

在Keras文档中进行嵌入 https：// keras.io/layers/embeddings/ ，对于 mask_zero 的解释是

In the Keras docs for Embedding https://keras.io/layers/embeddings/, the explanation given for mask_zero is

mask_zero：输入值0是否为应屏蔽的特殊填充值。这在使用可能需要可变长度输入的循环图层时很有用。如果为True，则模型中的所有后续层都需要支持屏蔽，否则将引发异常。结果，如果mask_zero设置为True，则词汇表中不能使用索引0（input_dim应等于| vocabulary | + 2）。

mask_zero: Whether or not the input value 0 is a special "padding" value that should be masked out. This is useful when using recurrent layers which may take variable length input. If this is True then all subsequent layers in the model need to support masking or an exception will be raised. If mask_zero is set to True, as a consequence, index 0 cannot be used in the vocabulary (input_dim should equal |vocabulary| + 2).

为什么input_dim必须是2 +词汇量？假设0被屏蔽并且无法使用，难道它不只是1 +字数吗？

Why does input_dim need to be 2 + number of words in vocabulary? Assuming 0 is masked and can't be used, shouldn't it just be 1 + number of words? What is the other extra entry for?

推荐答案

我相信那里的文档有些误导。在正常情况下，您正在映射 n 输入数据索引 [0，1，2，...，n-1] 到向量，因此您的 input_dim 应该与您拥有的元素数量一样

I believe the docs are a bit misleading there. In the normal case you are mapping your n input data indices [0, 1, 2, ..., n-1] to vectors, so your input_dim should be as many elements as you have

input_dim = len(vocabulary_indices)

一种等效的表达方式（但有点令人困惑） ，以及文档的操作方式是

An equivalent (but slightly confusing) way to say this, and the way the docs do, is to say

1 +输入数据中出现的最大整数索引。

1 + maximum integer index occurring in the input data.

input_dim = max(vocabulary_indices) + 1

如果启用屏蔽，则对值 0 的处理会有所不同，因此您增加 n 索引加1： [0、1、2，...，n-1，n] ，因此您需要

If you enable masking, value 0 is treated differently, so you increment your n indices by one: [0, 1, 2, ..., n-1, n], thus you need

input_dim = len(vocabulary_indices) + 1

或

input_dim = max(vocabulary_indices) + 2

文档变为esp

The docs become especially confusing here as they say

（input_dim应该等于 | vocabulary | + 2 ）

在这里我会解释 | x | 作为集合的基数（相当于 len（x）），但是作者的意思似乎是

where I would interpret |x| as the cardinality of a set (equivalent to len(x)), but the authors seem to mean

2 +输入数据中出现的最大整数索引。

2 + maximum integer index occurring in the input data.

这篇关于Keras嵌入层蒙版。为什么input_dim需要| vocalbulary | + 2？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！