如何使用 nltk 找出英语中是否存在某个单词 [英] How to find out wether a word exists in english using nltk

问题描述

我正在为这个问题寻找合适的解决方案.这个问题以前被问过很多次，我没有找到一个合适的答案.我需要使用NLTK中的语料库来检测一个词是否是英文词

我尝试过:

wordnet.synsets(word)

这不适用于许多常用词.使用英文单词列表并在文件中执行查找不是一种选择.使用附魔也不是一种选择.如果有其他库可以做同样的事情，请提供api的用法.如果没有，请提供一个包含所有英文单词的 nltk 语料库.

解决方案

NLTK 包括一些语料库，它们只不过是词表.Words Corpus 是来自 Unix 的/usr/share/dict/words 文件，被一些拼写检查器使用.我们可以使用它在文本语料库中查找异常或拼写错误的单词，如下所示:

def 不寻常的词(文本):text_vocab = set(w.lower() for w in text.split() if w.isalpha())english_vocab = set(w.lower() for w in nltk.corpus.words.words())不寻常 = text_vocab - english_vocab返回排序(不寻常)

在这种情况下，您可以使用 english_vocab 检查您的单词的成员资格.

<预><代码>>>>导入 nltk>>>english_vocab = set(w.lower() for w in nltk.corpus.words.words())>>>'a' in english_vocab真的>>>'this' in english_vocab真的>>>english_vocab 中的没什么"真的>>>'nothingg' in english_vocab错误的>>>english_vocab 中的语料库"真的>>>'Terminology'.lower() in english_vocab真的>>>english_vocab 中的排序"真的

I am looking for a proper solution to this question. This question has been asked many times before and I didn't find a single answer that suited. I need to use a corpus in NLTK to detect whether a word is an English word

I have tried to do :

wordnet.synsets(word)

This doesn't work for many common words. Using a list of words in English and performing lookup in a file is not an option. Using enchant is not an option either. If there is another library that can do the same, please provide the usage of the api. If not, please provide a corpus in nltk which has all the words in English.

解决方案

NLTK includes some corpora that are nothing more than wordlists. The Words Corpus is the /usr/share/dict/words file from Unix, used by some spell checkers. We can use it to find unusual or mis-spelt words in a text corpus, as shown in :

def unusual_words(text):
    text_vocab = set(w.lower() for w in text.split() if w.isalpha())
    english_vocab = set(w.lower() for w in nltk.corpus.words.words())
    unusual = text_vocab - english_vocab
    return sorted(unusual)

And in this case you can check the member ship of your word with english_vocab.

>>> import nltk
>>> english_vocab = set(w.lower() for w in nltk.corpus.words.words())
>>> 'a' in english_vocab
True
>>> 'this' in english_vocab
True
>>> 'nothing' in english_vocab
True
>>> 'nothingg' in english_vocab
False
>>> 'corpus' in english_vocab
True
>>> 'Terminology'.lower() in english_vocab
True
>>> 'sorted' in english_vocab
True

这篇关于如何使用 nltk 找出英语中是否存在某个单词的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！