nltk的分词器错误 [英] tokenizer errors with nltk

问题描述

我是Python的新手，正在尝试结合使用nltk进行学习.

I'm very new to Python, and am trying to learn in conjunction with using nltk.

我一直在跟踪一些示例并进行测试，但是由于python返回的错误，我似乎只能做些限制.

I've been following some examples and testing things out, but it seems I am very limited in what I can do due to errors being returned by python.

我知道已安装nltk并可以正常导入，因为此代码有效

I know nltk is installed and importing fine, because this code works

from nltk.sem import chat80
print chat80.items

但是，从nltk.tokenizer导入*"返回

However, 'from nltk.tokenizer import *' returns

'File "stdin", line1. 

使用任何类型的"TOKEN ="时，我都会遇到类似的错误，或者我正在猜测任何内容的标记化.

I get similar errors when using any sort of "TOKEN=" or I'm guessing tokenization of anything.

过去几天我已经多次安装了python，希望使用其他版本或更好的安装会有所帮助.

I've installed python many times in the last few days, hoping a different version or better install might help.

我在Windows7上使用activePython2.6遇到此错误，尽管我得到了类似的错误 与python 3.1 activePython3.1和Python 2.6一起使用. 以及使用Python 2.5的Mac OSx 10.5.

I'm getting this error on windows7 using activePython2.6, though I've gotten similar err ors with python 3.1 activePython3.1 and Python 2.6. as well as on Mac OSx 10.5 with Python 2.5.

mac正在通过导入错误:没有名为tokenizer的模块."来提供更多数据.

The mac is giving a bit more data with "Import Error: No module named tokenizer.

我只是尝试一些在线入门演示，甚至没有尝试编写自己的代码，而且我遇到的错误多于成功.

I'm just trying some of the introductory demos to nltk online, not even trying to write my own code yet, and I'm getting more errors than successes.

推荐答案

看起来nltp软件包没有tokenizer软件包.

Looks like the nltp package doesn't have a tokenizer package.

快速浏览NLTK网站，就会发现from nltp.tokenize import *是您所追求的.

A quick look on the NLTK website suggests that from nltp.tokenize import * is what you're after.

这篇关于nltk的分词器错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！