训练CNN时如何对输入word2vec进行微调 [英] how's the input word2vec get fine-tuned when training CNN

问题描述

当我阅读卷积神经网络用于句子分类"(Yoon Kim-纽约大学)时，我注意到该论文实现了"CNN-非静态"模型-该模型具有来自word2vec的预训练向量，所有单词(包括随机初始化的未知单词)，以及针对每个任务的预训练矢量都将进行微调. 因此，我只是不了解如何针对每个任务微调预训练向量.据我所知，输入向量是通过word2vec.bin(预训练)从字符串转换而来的，就像图像矩阵一样，在训练CNN时不会改变.那么，如果可以，怎么办?请帮帮我，非常感谢！

When I read the paper "Convolutional Neural Networks for Sentence Classification"-Yoon Kim-New York University, I noticed that the paper implemented the "CNN-non-static" model--A model with pre-trained vectors from word2vec,and all words— including the unknown ones that are randomly initialized, and the pre-trained vectors are fine-tuned for each task. So I just do not understand how the pre-trained vectors are fine-tuned for each task. Cause as far as I know, the input vectors, which are converted from strings by word2vec.bin(pre-trained), just like image matrix, which can not change during training CNN. So, if they can, HOW? Please help me out, Thanks a lot in advance!

推荐答案

词嵌入是神经网络的权重，因此可以在反向传播期间进行更新.

The word embeddings are weights of the neural network, and can therefore be updated during backpropagation.

例如 http://sebastianruder.com/word-embeddings-1/:

自然地，每个前馈神经网络都将词汇表中的单词作为输入并将它们作为矢量嵌入到较低维空间中，然后通过反向传播对其进行微调，因此必然产生单词嵌入作为权重.第一层，通常称为嵌入层.

Naturally, every feed-forward neural network that takes words from a vocabulary as input and embeds them as vectors into a lower dimensional space, which it then fine-tunes through back-propagation, necessarily yields word embeddings as the weights of the first layer, which is usually referred to as Embedding Layer.

这篇关于训练CNN时如何对输入word2vec进行微调的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！