如何使用CFStringTokenizer获得一个句子数组？ [英] How to get an array of sentences using CFStringTokenizer?

问题描述

我已经创建了这样的字符串标记器：

I've created an string tokenizer like this:

stringTokenizer = CFStringTokenizerCreate(
                         NULL
                         , (CFStringRef)str
                         , CFRangeMake(0, [str length])
                         , kCFStringTokenizerUnitSentence
                         , userLocale);

但是我现在如何从令牌生成器获取这些句子？ 《 CF String编程指南》未提及 CFStringTokenizer 或令牌（在PDF中进行了全文搜索）。

But how do I obtain those sentences now from the tokenizer? The CF String Programming Guide doesn't mention CFStringTokenizer or tokens (did a full-text search in the PDF).

推荐答案

下面是 CFStringTokenizer 用法：

CFStringRef string; // Get string from somewhere
CFLocaleRef locale = CFLocaleCopyCurrent();

CFStringTokenizerRef tokenizer = 
    CFStringTokenizerCreate(
        kCFAllocatorDefault
        , string
        , CFRangeMake(0, CFStringGetLength(string))
        , kCFStringTokenizerUnitSentence
        , locale);

CFStringTokenizerTokenType tokenType = kCFStringTokenizerTokenNone;
unsigned tokensFound = 0;

while(kCFStringTokenizerTokenNone !=
    (tokenType = CFStringTokenizerAdvanceToNextToken(tokenizer))) {
    CFRange tokenRange = CFStringTokenizerGetCurrentTokenRange(tokenizer);
    CFStringRef tokenValue =
        CFStringCreateWithSubstring(
            kCFAllocatorDefault
            , string
            , tokenRange);

  // Do something with the token
  CFShow(tokenValue);
  CFRelease(tokenValue);
  ++tokensFound;
}

// Clean up
CFRelease(tokenizer);
CFRelease(locale);

这篇关于如何使用CFStringTokenizer获得一个句子数组？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！