从句子生成N-gram [英] N-gram generation from a sentence

问题描述

如何生成n-gram的字符串，如：

How to generate an n-gram of a string like:

String Input="This is my car."

我想用这个输入生成n-gram：

I want to generate n-gram with this input:

Input Ngram size = 3

输出应该be：

This
is
my
car

This is
is my
my car

This is my
is my car

在Java中给出一些想法，如何实现它或者是否有可用的库。

Give some idea in Java, how to implement that or if any library is available for it.

我正在尝试使用这个NGramTokenizer 但它给出了n-gram的字符序列，我想要n-gram的单词序列。

I am trying to use this NGramTokenizer but its giving n-gram's of character sequence and I want n-grams of word sequence.

推荐答案

您正在寻找 ShingleFilter 。

更新：链接指向3.0.2版。这个类可能在较新版本的Lucene中使用不同的包。

Update: The link points to version 3.0.2. This class may be in different package in newer version of Lucene.

这篇关于从句子生成N-gram的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！