Python Gensim:如何使用 LDA 模型计算文档相似度? [英] Python Gensim: how to calculate document similarity using the LDA model?

问题描述

我有一个经过训练的 LDA 模型，我想从我训练模型的语料库中计算两个文档之间的相似度得分.在学习了所有 Gensim 教程和功能后，我仍然无法理解它.有人可以给我一个提示吗?谢谢！

解决方案

不知道这是否会有所帮助，但是，当使用实际文档作为查询时，我设法在文档匹配和相似性方面取得了成功的结果.

>

dictionary = corpora.Dictionary.load('dictionary.dict')语料库 = corpora.MmCorpus("corpus.mm")lda = models.LdaModel.load("model.lda") #运行在线lda的结果(训练)索引 = Similarities.MatrixSimilarity(lda[语料库])index.save("simIndex.index")docname = "docs/the_doc.txt"doc = open(docname, 'r').read()vec_bow = dictionary.doc2bow(doc.lower().split())vec_lda = lda[vec_bow]sims = 索引[vec_lda]sims = sorted(enumerate(sims), key=lambda item: -item[1])打印模拟人生

位于语料库中的所有文档与用作查询的文档之间的相似度分数将是每个 sim 的第二个索引.

I've got a trained LDA model and I want to calculate the similarity score between two documents from the corpus I trained my model on. After studying all the Gensim tutorials and functions, I still can't get my head around it. Can somebody give me a hint? Thanks!

解决方案

Don't know if this'll help but, I managed to attain successful results on document matching and similarities when using the actual document as a query.

dictionary = corpora.Dictionary.load('dictionary.dict')
corpus = corpora.MmCorpus("corpus.mm")
lda = models.LdaModel.load("model.lda") #result from running online lda (training)

index = similarities.MatrixSimilarity(lda[corpus])
index.save("simIndex.index")

docname = "docs/the_doc.txt"
doc = open(docname, 'r').read()
vec_bow = dictionary.doc2bow(doc.lower().split())
vec_lda = lda[vec_bow]

sims = index[vec_lda]
sims = sorted(enumerate(sims), key=lambda item: -item[1])
print sims

Your similarity score between all documents residing in the corpus and the document that was used as a query will be the second index of every sim for sims.

这篇关于Python Gensim:如何使用 LDA 模型计算文档相似度?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！