在lucene中使用点击荧光笔 [英] using hit highlighter in lucene
问题描述
关于使用apache lucene提供的点击荧光笔我有两个问题：
I have two questions regarding hit highlighter provided with apache lucene:
-
参见这个函数
你能解释一下令牌流参数的使用吗。
see this function could you explain the use of token stream parameter.
我有几个大的lucene文档包含许多字段，每个字段中都包含一些字符串。现在我找到了特定查询最相关的文档。现在发现此文档是因为查询中的多个单词可能与文档中的单词匹配。我想找出查询中的哪些单词导致了这个问题。因此，我计划使用Lucene Hit Highlighter。
示例：如果查询是皮肤医生德里，而标题为皮肤科医生的文件中包含皮肤和医生字样，那么在点击突出显示后我应该能够分离出皮肤和医生来自查询。我一直试图为此编写代码数周。无法得到我想要的东西。你可以帮帮我吗？
I have several large lucene document containing many fields and each field has some strings in it. Now I have found the most relevant document for a particular query. Now this document was found because several words in the query might have matched with the words in the document. I want to find out what words in the query caused this. So for this I plan to use Lucene Hit Highlighter. Example: if the query is "skin doctor delhi" and the document titled "dermatologist" contains the words "skin" and "doctor" then after hit highlighting i should be able to separate out "skin" and "doctor" from the query. I have been trying to write the code for this for several weeks now. Not able to get what i want. Could you help me please?
提前致谢。
更新：
当前方法：
我创建一个包含文档中所有单词的查询。
Current Approach: I create a query containing all the words in the document.
Field[] field = doc.getFields("description");
String desc = "";
for (int j = 0; j < field.length; ++j) {
desc += field[j].stringValue() + " ";
}
Query q = qp.parse(desc);
QueryScorer scorer = new QueryScorer(q, reader, "description");
Highlighter highlighter = new Highlighter(scorer);
String fragment = highlighter.getBestFragment(analyzer, "description", text);
它适用于小型文档，但不适用于大型文档。获得以下堆栈跟踪。
It works for small documents but does not work for large documents. The following stacktrace is obtained.
org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 1024
at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:152)
at org.apache.lucene.queryParser.QueryParser.getBooleanQuery(QueryParser.java:891)
at org.apache.lucene.queryParser.QueryParser.getBooleanQuery(QueryParser.java:866)
at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1213)
at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1167)
at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:182)
很明显，这种方法对于大型文档来说是不合理的。应该怎么做才能纠正这个问题？
It is obvious that the approach is unreasonable for large documents. What should be done to correct this?
BTW我正在使用FuzzyQuery匹配。
BTW I am using FuzzyQuery matching.
推荐答案
编辑：添加了一些关于explain（）的细节。
added some details about explain().
一些一般性介绍：
Lucene荧光笔用于查找来自命中的文本片段文档，
并突出显示与查询匹配的标记。
Some general introduction: The Lucene Highlighter is meant to find text snippets from a hit document, and to highlight tokens matching the query.
- 因此，TokenStream参数用于将命中文本分解为标记。然后，荧光笔的得分手会对每个标记进行评分，以便对片段进行评分并选择要突出显示的片段和标记。
- 我相信你做错了。如果您只想了解文档中匹配的查询字词，则应使用 explain（）方法。
基本上，在您实例化搜索者后，请使用：
- Therefore, The TokenStream parameter is used to break the hit text into tokens. The highlighter's scorer then scores each token, in order to score fragments and choose snippets and tokens to be highlighted.
- I believe you are doing it wrong. If all you want to do is understand which query terms were matched in the document, you should use the explain() method. Basically, after you have instantiated a searcher, use:
说明expl = searcher.explain （query，docId）;
String asText = expl.toString（）;
String asHtml = expl.toHtml（）;
docId是搜索结果中的原始文档ID。
docId is the raw document id from the search results.
只有在需要片段和/或突出显示时，才应使用荧光笔。如果您仍想使用荧光笔，请按照 Nicholas Hrychan的建议 。但要注意，他描述了Lucene 2.4.1 API - 如果你使用更高级的版本，你应该使用QueryScorer，他说SpanScorer。
Only if you do need the snippets and/or highlights, you should use the Highlighter. If you still want to use the highlighter, follow Nicholas Hrychan's advice. Beware, though, as he describes the Lucene 2.4.1 API - If you use a more advanced version, you should use "QueryScorer" where he says "SpanScorer" .
这篇关于在lucene中使用点击荧光笔的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
