Solr 4.4:StopFilterFactory和enablePositionIncrements [英] Solr 4.4: StopFilterFactory and enablePositionIncrements

问题描述

在尝试从Solr 4.3.0升级到Solr 4.4.0时，我遇到了此异常:

While attempting to upgrade from Solr 4.3.0 to Solr 4.4.0 I ran into this exception:

 java.lang.IllegalArgumentException: enablePositionIncrements=false is not supported anymore as of Lucene 4.4 as it can create broken token streams

这导致我此问题.我需要能够匹配查询，而与插入的停用词无关(该停用词过去曾与enablePositionIncrements ="true"配合使用).例如:"bar of foo"将找到与"foo bar"，"bar of foo"和"bar of foo"匹配的文档.在4.4.0中弃用了此选项后，我不清楚如何维护相同的功能.

which led me to this issue. I need to be able to match queries irrespective of intervening stopwords (which used to work with enablePositionIncrements="true"). For instance: "foo of the bar" would find documents matching "foo bar", "foo of bar", and "foo of the bar". With this option deprecated in 4.4.0 I'm not clear on how to maintain the same functionality.

程序包javadoc 添加:

如果选定的分析器过滤了停用词"is"和"the"，则对于包含字符串"blue is the sky"的文档，仅标记标记"blue"，"sky"，并带有position(天空")= 3 +位置(蓝色").现在，短语查询蓝色是天空"将找到该文档，因为同一分析器会从该查询中过滤相同的停用词.但是短语查询"blue sky"将找不到该文档，因为"blue"和"sky"之间的位置增量仅为1.

If the selected analyzer filters the stop words "is" and "the", then for a document containing the string "blue is the sky", only the tokens "blue", "sky" are indexed, with position("sky") = 3 + position("blue"). Now, a phrase query "blue is the sky" would find that document, because the same analyzer filters the same stop words from that query. But the phrase query "blue sky" would not find that document because the position increment between "blue" and "sky" is only 1.

如果此行为不符合应用程序的需求，则需要将查询解析器配置为在生成词组查询时不考虑位置增量.

If this behavior does not fit the application needs, the query parser needs to be configured to not take position increments into account when generating phrase queries.

但是没有提到如何实际配置查询解析器来做到这一点.当Solr迈向5.0时，有人知道如何处理此问题吗?

But there's no mention of how to actually configure the query parser to do this. Does anyone know how to deal with this issue as Solr moves toward 5.0?

推荐答案

我在RemoveTokenGapsFilterFactory的网络实现中找到了某处

I found somewhere on the net implementation of RemoveTokenGapsFilterFactory

public final class RemoveTokenGapsFilter extends TokenFilter {

    private final PositionIncrementAttribute posIncrAttribute = addAttribute(PositionIncrementAttribute.class);

    public RemoveTokenGapsFilter(TokenStream input) {
        super(input);
    }

    @Override
    public boolean incrementToken() throws IOException {

        if (input.incrementToken()) {
            posIncrAttribute.setPositionIncrement(1);
            return true;
        }

        return false;
    }
}

这篇关于Solr 4.4:StopFilterFactory和enablePositionIncrements的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！