Antlr 4.5 运行时解析器错误 [英] Antlr 4.5 parser error during runtime

问题描述

为了学习目的，我正在为语言编程构建简单的语法.

I'm building simple grammar for programming laguange for learning purposes.

我遇到了对我来说毫无意义的奇怪错误.

I run into strange error that make no sense to me.

line 1:0 missing {'void', 'int', 'bool', 'string', 'union'} at 'void'

我正在使用此语法中的预构建词法分析器和解析器:

I'm using prebuild lexer and parser from this grammar:

grammar ProgrammingLanguage;

function_definition
    : type_specifier IDENTIFIER '(' parameter_list_opt ')' compound_statement
    ;

type_specifier
    : VOID
    | INT
    | BOOL
    | STRING
    | UNION
    ;

compound_statement
    : '{' declaration_list statement_list '}'
    ;

statement_list
    : statement
    | statement statement_list
    |
    ;

statement
    : compound_statement
    | selection_statement
    | while_statement
    | jump_statement
    | expression_statement
    | comment_statement
    ;

comment_statement
    : COMMENT_START COMMENT
    ;

selection_statement
    : IF '(' expression ')' compound_statement
    | IF '(' expression ')' compound_statement ELSE compound_statement
    | SWITCH '(' expression ')' compound_statement
    ;

expression_statement
    : ';'
    | expression ';'
    ;

jump_statement
    : BREAK ';'
    | CONTINUE ';'
    ;

while_statement
    : WHILE '(' expression ')' compound_statement
    ;

primary_expression
    : IDENTIFIER
    | CONSTANT
    | '(' expression ')'
    | IDENTIFIER '(' primary_expression_list ')'
    ;

primary_expression_list
    : primary_expression
    | primary_expression primary_expression_list
    |
    ;

expression
    : logical_or_expression
    | additive_expression
    ;

logical_or_expression
    : logical_and_expression
    | logical_or_expression '||' logical_and_expression
    ;

logical_and_expression
    : compare_expression
    | logical_and_expression '&&' compare_expression
    ;

compare_expression
    : primary_expression compare_op primary_expression
    | primary_expression
    ;

compare_op
    : '<'
    | '>'
    | '=='
    | '!='
    | '<='
    | '>='
    ;

additive_expression
    : multiplicative_expression
    | additive_expression '+' multiplicative_expression
    | additive_expression '-' multiplicative_expression
    ;

multiplicative_expression
    : primary_expression
    | multiplicative_expression '*' primary_expression
    | multiplicative_expression '/' primary_expression
    | multiplicative_expression '%' primary_expression
    ;

assignment_expression
    : IDENTIFIER '=' expression
    ;

id_list
    : IDENTIFIER
    | IDENTIFIER ',' id_list
    ;

declaration
    : type_specifier id_list ';'
    ;

parameter_list_opt
    : parameter_list
    |
    ;

parameter_list
    : type_specifier IDENTIFIER
    | type_specifier IDENTIFIER ',' parameter_list
    ;

declaration_list
    : declaration
    | declaration declaration_list
    |
    ;

/**------------------------------------------------------------------
 * LEXER RULES
 *------------------------------------------------------------------
 */
WHILE   : 'while' ;

BREAK   : 'break' ;
CONTINUE    : 'continue' ;
SWITCH  : 'switch' ;

IF  : 'if' ;
ELSE    : 'else' ;

COMMENT_START   : '//' ;

IDENTIFIER  :   ('a'..'z'|'A'..'Z')('0'..'9'|'a'..'z'|'A'..'Z')*;
CONSTANT    :   FALSE|TRUE|STRING_VALUE|INT_VALUE;
STRING_VALUE : '"'COMMENT'"';
COMMENT : ('0'..'9'|'a'..'z'|'A'..'Z')*;
INT_VALUE : ('0'..'9')+;
FALSE : 'false';
TRUE : 'true';

VOID : 'void';
INT : 'int';
BOOL : 'bool';
STRING : 'string';
UNION : 'union';

WS :    (' '|'\t'|'\n'|'\r')+ -> skip;

我正在用这个 java 代码解析:

And I'm parsing with this java code:

import org.antlr.v4.runtime.*;
import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.ParseTreeWalker;

import java.io.IOException;

public class Main {
    public static void main(String[] args) throws IOException {
        ProgrammingLanguageLexer lexer = new ProgrammingLanguageLexer(new ANTLRFileStream("input.txt"));
        ProgrammingLanguageParser parser = new ProgrammingLanguageParser(new CommonTokenStream(lexer));
        ParseTree tree = parser.function_definition();
        ParseTreeWalker.DEFAULT.walk(new ProgrammingLanguageBaseListener(), tree);
    }
}

最后是我试图解析的字符串:

And finally string, that I'm trying to parse:

void power () {}

推荐答案

该错误消息意味着包含值 'void' 的预期标记类型与从输入中使用字符串 'void' 产生的实际标记类型不匹配.查看您的词法分析器规则表明，IDENTIFIER 规则正在使用输入字符串void"，从而生成 IDENTIFIER 类型的标记，而不是 VOID.

The error message means that the expected token type containing the value 'void' does not match the actual token type produced by consuming the string 'void' from the input. Looking at your lexer rules suggests that the input string 'void' is being consumed by the IDENTIFIER rule, producing a token of type IDENTIFIER, not VOID.

一般来说，匹配最长输入字符串的词法分析器规则获胜.对于具有相同匹配长度的两个(或多个)规则，首先列出的规则获胜.将所有关键字规则移至 IDENTIFIER 规则之上.

In general, the lexer rule that matches longest input string wins. For two (or more) rules with the same match length, the first listed wins. Move all of your keyword rules above the IDENTIFIER rule.

一个有用的单元测试表单将转储 lex 的标记并显示匹配的实际标记类型.类似的东西:

A helpful unit test form will dump the lex'd tokens and show the actual token types matched. Something like:

CommonTokenStream tokens = ...
tokens.fill();
StringBuilder sb = new StringBuilder();
for (Token token : tokens.getTokens()) {
    sb.append(((YourCustomTokenType) token).toString());
}
System.out.print(sb.toString());

Token.toString() 方法通常已经足够好了.覆盖您的令牌子类以满足您自己的需要.

The Token.toString() method is usually good enough. Override in your token subclass to fit your own needs.

这篇关于Antlr 4.5 运行时解析器错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！