针对语义相似性对Bert进行了微调 [英] Bert fine-tuned for semantic similarity

问题描述

我想应用微调的Bert来计算句子之间的语义相似度. 我搜索了很多网站，但在下游几乎找不到.

I would like to apply fine-tuning Bert to calculate semantic similarity between sentences. I search a lot websites, but I almost not found downstream about this.

我刚刚找到了 STS基准. 我想知道是否可以使用STS基准数据集来训练微调的bert模型，并将其应用于我的任务. 合理吗?

I just found STS benchmark. I wonder if I can use STS benchmark dataset to train a fine-tuning bert model, and apply it to my task. Is it reasonable?

据我所知，有很多计算相似度的方法，包括余弦相似度，皮尔逊相关度，曼哈顿距离等. 如何选择语义相似性?

As I know, there are a lot method to calculate similarity including cosine similarity, pearson correlation, manhattan distance, etc. How choose for semantic similarity?

推荐答案

作为前面的一般性评论，我想强调一下，此类问题可能不被视为Stackoverflow的主题，请参见 AI Stackexchange 或已通过交叉验证.

As a general remark ahead, I want to stress that this kind of question might not be considered on-topic on Stackoverflow, see How to ask. There are, however, related sites that might be better for these kinds of questions (no code, theoretical PoV), namely AI Stackexchange, or Cross Validated.

如果您查看的是颇受欢迎Mueller和Thyagarajan撰写的关于该领域论文的研究是关于在LSTM上学习句子相似性，他们使用了密切相关的数据集( SICK数据集)，并在2014年与STS基准测试并列.

If you look at a rather popular paper in the field by Mueller and Thyagarajan, which is concerned with learning sentence similarity on LSTMs, they use a closely related dataset (the SICK dataset), which is also hosted by the SemEval competition, and ran alongside the STS benchmark in 2014.

其中任何一个都应该合理设置，但是STS已经运行了多年，因此可用的培训数据量可能会更大.

Either one of those should be a reasonable set to fine-tune on, but STS has run over multiple years, so the amount of available training data might be larger.

作为该主题的出色入门，我也强烈推荐Adrien Sieg撰写的Medium文章(请参阅

As a great primer on the topic, I can also highly recommend the Medium article by Adrien Sieg (see here, which comes with an accompanied GitHub reference.

对于语义相似性，我估计您最好对神经网络进行微调(或训练)，因为您提到的大多数经典相似性度量都更加着重于标记相似性(因此，句法相似性，尽管不一定如此).另一方面，语义含义有时可能在单个单词(可能是一个否定词，或者两个单词的互换句子位置)上有很大的不同，这很难用静态方法来解释或评估.

For semantic similarity, I would estimate that you are better of with fine-tuning (or training) a neural network, as most classical similarity measures you mentioned have a more prominent focus on the token similarity (and thus, syntactic similarity, although not even that necessarily). Semantic meaning, on the other hand, can sometimes differ wildly on a single word (maybe a negation, or the swapped sentence position of two words), which is difficult to interpret or evaluate with static methods.

这篇关于针对语义相似性对Bert进行了微调的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！