Solr中的词频 [英] Word frequency in Solr

问题描述

我正在尝试使用solr获得单词的频率.当我给出此查询时:

localSolr/solr/select?q=someQuery&rows=0&facet=true&facet.field=content&wt=xml

solr给我类似的频率

<lst name="facet_counts">
<lst name="facet_queries"/>
<lst name="facet_fields">
<lst name="content">
<int name="word1">24</int>
<int name="word2">12</int>
<int name="word3">8</int>

但是当我数数单词时；我发现word2的实际计数值为13.Solr在字段中将相同的单词计数为一个.

例如；

字段文本由； word2 word5 word7 word9 word2. Solr不返回word2的计数2，而是返回1.它为下面两个句子的word2计数返回1；

word2 word10 word11 word12
word2 word9 word7 word2 word23

因此频率返回错误.我检查了构面字段，但没有找到合适的参数.我该如何解决它以便使句子中的单词数相同?

schema.xml的相关部分:

<fieldType name="text_tr" class="solr.TextField" positionIncrementGap="100">
    <field name="content" type="text_tr" stored="true" indexed="true" multiValued="true"/>
    <copyField source="content" dest="text"/>
    <field name="text" type="text_tr" stored="false" indexed="true" multiValued="true"/>

解决方案

如果您要针对的字段是多值的，则方面中的每个单词都会获得正确的计数

我忘了提一件事:术语向量组件将带您到需要的地方

在查询中， tv.tf 将为您提供每个术语的词频，而 tv.fl 告诉solr应该在哪个字段上计算频率

注意事项，这会使您的索引编制时间比现在慢(又称:您必须尝试)

I am trying to get frequency of words using solr. When I give this query :

localSolr/solr/select?q=someQuery&rows=0&facet=true&facet.field=content&wt=xml

solr gives me the frequencies like;

<lst name="facet_counts">
<lst name="facet_queries"/>
<lst name="facet_fields">
<lst name="content">
<int name="word1">24</int>
<int name="word2">12</int>
<int name="word3">8</int>

But when I count the words; I find that word2's actual count number is 13. Solr counts same words in the field as one.

For example;

field text consists; word2 word5 word7 word9 word2. Solr doesn't return word2's count number 2 instead it returns 1. It returns 1 for the count of word2 for the both sentences below;

word2 word10 word11 word12
word2 word9 word7 word2 word23

So frequencies return wrongly. I have checked facet fields but didn't find the proper parameter for that. How can I fix it so that it counts same words in sentence?

edit : relevant part of schema.xml :

<fieldType name="text_tr" class="solr.TextField" positionIncrementGap="100">
    <field name="content" type="text_tr" stored="true" indexed="true" multiValued="true"/>
    <copyField source="content" dest="text"/>
    <field name="text" type="text_tr" stored="false" indexed="true" multiValued="true"/>

解决方案

if the field you're faceting on is multivalued, then each word in the facet gets the proper count

i forgot to mention one thing: Term Vector Component will get you where you need

in the query, tv.tf will give you the term frequency for each term, while tv.fl tells solr on which fields the frequency should be calculated

NB this makes your indexing time slower than now (aka: you have to try it)

这篇关于Solr中的词频的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！