单个单词的空间词形还原 [英] Spacy lemmatization of a single word

问题描述

我正在尝试获取单个单词的词形还原版本.有没有办法使用spacy"(神奇的python NLP库)来做到这一点.

I am trying to get the lemmatized version of a single word. Is there a way using "spacy" (fantastic python NLP library) to do this.

以下是我尝试过的代码，但不起作用):

Below is the code I have tried but this does not work):

from spacy.lemmatizer import Lemmatizer
from spacy.lookups import Lookups
lookups = Lookups()
lemmatizer = Lemmatizer(lookups)
word = "ducks"
lemmas = lemmatizer.lookup(word)
print(lemmas)

我希望的结果是ducks"(复数)这个词会导致duck"(单数).不幸的是，返回了鸭子"(复数).

The result I was hoping for was that the word "ducks" (plural) would result in "duck" (singular). Unfortunately, "ducks" (plural) is returned.

有没有办法做到这一点?

Is there a way of doing this?

注意:我意识到我可以处理来自文档 (nlp(document)) 的整个单词字符串，然后找到所需的标记，然后获取其引理 (token.lemma_)，但是我需要的单词词形还原有点动态，不能作为大文档处理.

NOTE: I realize that I could process an entire string of words from a document (nlp(document)) and then find the required token and then get its lemma (token.lemma_), but the word(s) I need to lemmatize are somewhat dynamic and are not able to be processed as a large document.

推荐答案

如果您想对单个标记进行词形还原，请尝试使用简化的文本处理库 TextBlob:

If you want to lemmatize single token, try the simplified text processing lib TextBlob:

from textblob import TextBlob, Word
# Lemmatize a word
w = Word('ducks')
w.lemmatize()

输出

> duck

或NLTK

import nltk
from nltk.stem import SnowballStemmer
stemmer = nltk.stem.SnowballStemmer('english')
stemmer.stem('ducks')

输出

> duck

否则您可以继续使用 spaCy，但在禁用 parser 和 NER 管道组件后:

Otherwise you can keep using spaCy, but after disabling parser and NER pipeline components:

• 首先下载一个 12M 的小模型(在 OntoNotes 上训练的英文多任务 CNN)

$ python -m spacy download en_core_web_sm

• Python 代码

import spacy
nlp = spacy.load('en_core_web_sm', disable=['parser', 'ner']) # just keep tagger for lemmatization
" ".join([token.lemma_ for token in nlp('ducks')])

输出

> duck

这篇关于单个单词的空间词形还原的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！