FreqDist 与 NLTK [英] FreqDist with NLTK
问题描述
NLTK 在 python 中有一个函数 FreqDist 它为您提供文本中单词的频率.我试图将我的文本作为参数传递,但结果采用以下形式:
NLTK in python has a function FreqDist which gives you the frequency of words within a text. I am trying to pass my text as an argument but the result is of the form:
[' ', 'e', 'a', 'o', 'n', 'i', 't', 'r', 's', 'l', 'd','h', 'c', 'y', 'b', 'u', 'g', '
', 'm', 'p', 'w', 'f', ',', 'v', '.', "'", 'k', 'B', '"', 'M', 'H', '9', 'C', '-', 'N', 'S', '1', 'A', 'G', 'P', 'T', 'W', '[', ']', '(', ')', '0', '7', 'E', 'J', 'O', 'R', 'j', 'x']
[' ', 'e', 'a', 'o', 'n', 'i', 't', 'r', 's', 'l', 'd', 'h', 'c', 'y', 'b', 'u', 'g', '
', 'm', 'p', 'w', 'f', ',', 'v', '.', "'", 'k', 'B', '"', 'M', 'H', '9', 'C', '-', 'N', 'S', '1', 'A', 'G', 'P', 'T', 'W', '[', ']', '(', ')', '0', '7', 'E', 'J', 'O', 'R', 'j', 'x']
而在 NLTK 网站的示例中,结果是整个单词而不仅仅是字母.我是这样做的:
whereas in the example in the NLTK website the result was whole words not just letters. Im doing it this way:
file_y = open(fileurl)
p = file_y.read()
fdist = FreqDist(p)
vocab = fdist.keys()
vocab[:100]
你知道我有什么问题吗?谢谢!
DO you know what I have wrong pls? Thanks!
推荐答案
FreqDist
需要可迭代的标记.字符串是可迭代的——迭代器产生每个字符.
FreqDist
expects an iterable of tokens. A string is iterable --- the iterator yields every character.
首先将您的文本传递给标记器,然后将标记传递给 FreqDist
.
Pass your text to a tokenizer first, and pass the tokens to FreqDist
.
这篇关于FreqDist 与 NLTK的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!