在神经网络中什么是投影层? [英] What is a projection layer in the context of neural networks?

问题描述

我目前正在尝试理解 word2vec 神经网络学习算法背后的体系结构，用于根据上下文将单词表示为矢量.

I am currently trying to understand the architecture behind the word2vec neural net learning algorithm, for representing words as vectors based on their context.

在阅读 Tomas Mikolov的论文之后，我发现了他所定义的投影层.尽管这个词在提到 word2vec 时被广泛使用，但我找不到它在神经网络环境中的确切定义.

After reading Tomas Mikolov paper I came across what he defines as a projection layer. Even though this term is widely used when referred to word2vec, I couldn't find a precise definition of what it actually is in the neural net context.

我的问题是，在神经网络环境中，什么是投影层?它是给与先前节点的链接共享相同权重的隐藏层的名称吗?它的单位实际上具有某种激活功能吗?

My question is, in the neural net context, what is a projection layer? Is it the name given to a hidden layer whose links to previous nodes share the same weights? Do its units actually have an activation function of some kind?

可以在本教程，该教程还引用了第67页周围的投影层.

Another resource that also refers more broadly to the problem can be found in this tutorial, which also refers to a projection layer around page 67.

推荐答案

投影层将n元语法上下文的离散单词索引映射到连续的向量空间.

The projection layer maps the discrete word indices of an n-gram context to a continuous vector space.

如该论文中所述

共享投影层，以便对于多次包含相同单词的上下文，应用相同的权重集以形成投影矢量的每个部分. 该组织有效地增加了可用于训练投影层权重的数据量，因为每个上下文训练模式的每个单词都分别对权重值做出了贡献.

The projection layer is shared such that for contexts containing the same word multiple times, the same set of weights is applied to form each part of the projection vector. This organization effectively increases the amount of data available for training the projection layer weights since each word of each context training pattern individually contributes changes to the weight values.

此图显示了简单的拓扑结构，如何通过复制投影层权重矩阵中的列来有效地组装投影层的输出.

this figure shows the trivial topology how the output of the projection layer can be efficiently assembled by copying columns from the projection layer weights matrix.

现在，隐藏层:

隐藏层处理投影层的输出，并使用 拓扑配置文件中指定的神经元数.

The hidden layer processes the output of the projection layer and is also created with a number of neurons specified in the topology configuration file.

编辑:对图中发生的情况的解释

Edit: An explanation of what is happening in the diagram

投影层中的每个神经元都由与词汇表大小相等的权重表示.投影层通过不使用非线性激活函数而不同于隐藏层和输出层.它的目的仅仅是提供一种有效的方法，将给定的n-gram上下文投影到减少的连续向量空间上，以便通过训练对这些向量进行分类的隐藏层和输出层进行后续处理.给定输入向量元素的一或零性质，具有索引i的特定单词的输出仅是投影层权重训练矩阵的第i列(其中矩阵的每一行代表单个神经元的权重) ).

Each neuron in the projection layer is represented by a number of weights equal to the size of the vocabulary. The projection layer differs from the hidden and output layers by not using a non-linear activation function. Its purpose is simply to provide an efficient means of projecting the given n- gram context onto a reduced continuous vector space for subsequent processing by hidden and output layers trained to classify such vectors. Given the one-or-zero nature of the input vector elements, the output for a particular word with index i is simply the ith column of the trained matrix of projection layer weights (where each row of the matrix represents the weights of a single neuron).

这篇关于在神经网络中什么是投影层?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！