使用Lucene 6阻止英语单词 [英] Stemming english words using Lucene 6

问题描述

我正在寻找使用Lucene 6.5阻止英语单词的方法.我已经看到了许多使用Lucene实现此目的的示例.但是，到目前为止，我所看到的示例似乎正在使用旧版本的Lucene，并且无法使用Lucene 6复制该版本.

I'm looking to stem English words using Lucene 6.5. I've seen quite a number of examples of using Lucene to achieve this. However, the examples I've seen so far seem to be using old versions of Lucene and replicating the same using Lucene 6 has not been possible.

一个典型的例子是这个.建议和接受的解决方案使用org.apache.lucene.analysis.PorterStemmer，它似乎与Lucene 6不在同一软件包中.

A case in point is this one. The suggested and accepted solutions use org.apache.lucene.analysis.PorterStemmer which doesn't seem to be in the same package in Lucene 6.

更新:我发现PorterStemmer词干提取器的当前完整路径为org.apache.lucene.analysis.en.PorterStemFilter.另外，需要依赖项"org.apache.lucene" % "lucene-queryparser" % "6.5.0".

UPDATE: I've found out that the current full path for the PorterStemmer stemmer is org.apache.lucene.analysis.en.PorterStemFilter. Additionally, one needs the dependency "org.apache.lucene" % "lucene-queryparser" % "6.5.0".

我正在研究词干列表.而且我认为我将把这个问题仅针对词干，因为我看到的使用该词干提取器的示例似乎不适用于当前版本的Lucene(6.5.0版)，也无法对其进行编译.

I'm now working on stemming a list of words. And I think I'll just pivot this question to word stemming since the examples I've seen using this stemmer do not seem to work nor compile with the current version of Lucene (ver 6.5.0).

推荐答案

我终于找到了一种使用Lucene 6阻止单词的方法:

I finally found a way to stem words using Lucene 6:

public List<String> stem(String term) throws Exception {
    Analyzer analyzer = new StandardAnalyzer();
    TokenStream result = analyzer.tokenStream(null, term);
    result = new PorterStemFilter(result);
    result = new StopFilter(result, StopAnalyzer.ENGLISH_STOP_WORDS_SET);
    CharTermAttribute resultAttr = result.addAttribute(CharTermAttribute.class);
    result.reset();

    List<String> tokens = new ArrayList<>();
    while (result.incrementToken()) {
        tokens.add(resultAttr.toString());
    }
    return tokens;
}

使用输入字符串term调用此方法，将返回从输入字符串生成的字符串标记的列表.另外，该方法从输入中删除停用词.我将其留在这里，希望对您有所帮助.

Calling this method with an input string, term, will return a list of string tokens generated from the input string. The method, additionally, removes stop words from the input. I'll leave this here in the hope that it'll be helpful to someone.

这篇关于使用Lucene 6阻止英语单词的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！