在lucene中获取两个文档之间的余弦相似度 [英] get cosine similarity between two documents in lucene

问题描述

我已经在 Lucene 中建立了一个索引.我想不指定查询，只是为了获得索引中两个文档之间的分数(余弦相似度或其他距离?).

i have built an index in Lucene. I want without specifying a query, just to get a score (cosine similarity or another distance?) between two documents in the index.

例如，我从以前打开的 IndexReader 中获取 id 为 2 和 4 的文档.文档 d1 = ir.document(2);文档 d2 = ir.document(4);

For example i am getting from previously opened IndexReader ir the documents with ids 2 and 4. Document d1 = ir.document(2); Document d2 = ir.document(4);

如何获得这两个文档之间的余弦相似度?

How can i get the cosine similarity between these two documents?

谢谢

推荐答案

索引时，可以选择存储词频向量.

When indexing, there's an option to store term frequency vectors.

在运行时，使用 IndexReader.getTermFreqVector() 查找两个文档的词频向量，并使用 IndexReader.docFreq() 查找每个词的文档频率数据.这将为您提供计算两个文档之间的余弦相似度所需的所有组件.

During runtime, look up the term frequency vectors for both documents using IndexReader.getTermFreqVector(), and look up document frequency data for each term using IndexReader.docFreq(). That will give you all the components necessary to calculate the cosine similarity between the two docs.

一种更简单的方法可能是将文档 A 作为查询提交(将所有单词作为 OR 词添加到查询中，按词频提升每个词)并在结果集中查找文档 B.

An easier way might be to submit doc A as a query (adding all words to the query as OR terms, boosting each by term frequency) and look for doc B in the result set.

这篇关于在lucene中获取两个文档之间的余弦相似度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！