如何在python输入文件中标记自然英语文本? [英] How to tokenize natural English text in an input file in python?

问题描述

我要tokenize input file in python请建议我我是python的新用户.

I want to tokenize input file in python please suggest me i am new user of python .

我阅读了一些有关正则表达式的内容，但仍然有些困惑，因此请提出相同的任何链接或代码概述.

I read the some thng about the regular expression but still some confusion so please suggest any link or code overview for the same.

推荐答案

尝试如下操作:

import nltk
file_content = open("myfile.txt").read()
tokens = nltk.word_tokenize(file_content)
print tokens

NLTK教程也充满了易于遵循的示例: http: //nltk.googlecode.com/svn/trunk/doc/book/ch03.html

The NLTK tutorial is also full of easy to follow examples: http://nltk.googlecode.com/svn/trunk/doc/book/ch03.html

这篇关于如何在python输入文件中标记自然英语文本?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！