在lucene中获得两个文档之间的余弦相似度 [英] get cosine similarity between two documents in lucene

问题描述

我在Lucene中建立了索引.我想要不指定查询，而只是获得索引中两个文档之间的分数(余弦相似度或另一个距离?).

i have built an index in Lucene. I want without specifying a query, just to get a score (cosine similarity or another distance?) between two documents in the index.

例如，我从先前打开的IndexReader ir中获取ID为2和4的文档. 文档d1 = ir.document(2); 文档d2 = ir.document(4);

For example i am getting from previously opened IndexReader ir the documents with ids 2 and 4. Document d1 = ir.document(2); Document d2 = ir.document(4);

如何获得这两个文档之间的余弦相似度?

How can i get the cosine similarity between these two documents?

谢谢

推荐答案

建立索引时，可以选择存储项频率向量.

When indexing, there's an option to store term frequency vectors.

在运行时，使用IndexReader.getTermFreqVector()查找两个文档的术语频率向量，并使用IndexReader.docFreq()查找每个术语的文档频率数据.这将为您提供计算两个文档之间的余弦相似度所需的所有组件.

During runtime, look up the term frequency vectors for both documents using IndexReader.getTermFreqVector(), and look up document frequency data for each term using IndexReader.docFreq(). That will give you all the components necessary to calculate the cosine similarity between the two docs.

一种更简单的方法可能是将文档A提交为查询(将所有单词作为OR词添加到查询中，按词频提高每个词的词频)，然后在结果集中查找doc B.

An easier way might be to submit doc A as a query (adding all words to the query as OR terms, boosting each by term frequency) and look for doc B in the result set.

这篇关于在lucene中获得两个文档之间的余弦相似度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！