如何将文本拆分成句子? [英] How can I split a text into sentences?

问题描述

我有一个文本文件.我需要得到一个句子列表.

I have a text file. I need to get a list of sentences.

如何实现?有很多微妙之处，例如在缩写中使用点.

How can this be implemented? There are a lot of subtleties, such as a dot being used in abbreviations.

我的旧正则表达式效果不佳:

My old regular expression works badly:

re.compile('(\. |^|!|\?)([A-Z][^;↑\.<>@\^&/\[\]]*(\.|!|\?) )',re.M)

推荐答案

The Natural Language Toolkit (nltk.org) 有你需要的.此群组发帖表明这样做了:

The Natural Language Toolkit (nltk.org) has what you need. This group posting indicates this does it:

import nltk.data

tokenizer = nltk.data.load('tokenizers/punkt/english.pickle')
fp = open("test.txt")
data = fp.read()
print '\n-----\n'.join(tokenizer.tokenize(data))

(我没试过！)

这篇关于如何将文本拆分成句子?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！