ANTLR:词法分析器无法识别令牌 [英] ANTLR: Lexer does not recognize token

问题描述

给定以下词法分析器语法:

Given the following Lexer grammar:

lexer grammar CodeTableLexer;

CodeTabHeader   : '[code table 1.0]';
Code            : 'code';
Table           : 'table';
End             : 'end';
Row             : 'row';
Naming          : 'naming';
Dfltlang        : 'dfltlang';
Language        : 'english' | 'german' | 'french' | 'italian' | 'spanish';
Null            : 'null';

Number
    : Int ('.' Digit*)?
    ;

Identifier
    : ('a'..'z' | 'A'..'Z' | '_') ('a'..'z' | 'A'..'Z' | '_' | '$' | '.' | Digit)*
    ;

String
@after {
    setText(getText().substring(1, getText().length() - 1).replaceAll("\\\\(.)", "$1"));
}
    : '"' (~('"'))* '"'
    ;

Comment
    : '--' ~('\r' | '\n')* { skip(); }
    | '/*' .* '*/'         { skip(); }
    ;

Space
    : (' ' | '\t' | '\r' | '\n' | '\u000C') { skip(); }
    ;

fragment Int
    : '1'..'9'
    | '0'
    ;

fragment Digit
    : '0'..'9'
    ;

... 以下解析器语法:

... the following parser grammar:

parser grammar CodeTableParser;

parse
    : header^ begin (row)* end EOF
    ;

header
    : CodeTabHeader
    ;

begin
    : Code Table Identifier row
    ;

row
    : Row (Number | Identifier) value
    ;

value
    : (Identifier (Number | Identifier | String))+
        (Naming Dfltlang String (Language String)*)?
    ;

end
    : End Code Table
    ;

... 最后是以下要解析的源:

... and finally the following source to parse:

[code table 1.0]

code table my_table
row 1
    id              "my_id_1"
    name            "my_name_1"
    descn           "my_description_1"
    naming
        dfltlang    "My description 1"
        english     "My description 1"
        german      "Meine Beschreibung 1"

row 2
    id              "my_id_2"
    name            "my_name_2"
    descn           "my_description_2"
    naming
        dfltlang    "My description 2"
        english     "My description 2"
        german      "Meine Beschreibung 2"

end code table

我总是收到错误信息

line 1:0 extraneous input '[code table 1.0]' expecting CodeTabHeader

我有点困惑……在我的词法分析器中，token CodeTabHeader 被定义为‘[code table 1.0]’；我错过了什么吗?任何帮助将不胜感激.

I’m a bit confused… token CodeTabHeader is defined as ‘[code table 1.0]’ in my lexer; am I missing something? Any help would be really appreciated.

谢谢，J3d

推荐答案

如果将词法分析器和解析器拆分为两个不同的文件，请在解析器的 options-block 中明确指出它应该使用哪些标记使用:

If you split the lexer and parser in two different files, explicitly indicate in the options-block of your parser which tokens it should use:

options {
  tokenVocab=CodeTableLexer;
}

这篇关于ANTLR:词法分析器无法识别令牌的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！