运行时期间Antlr 4.5解析器错误 [英] Antlr 4.5 parser error during runtime

问题描述

我正在构建简单的语法来编写laguange用于学习目的。

I'm building simple grammar for programming laguange for learning purposes.

我遇到了一些对我没用的奇怪错误。

I run into strange error that make no sense to me.

line 1:0 missing {'void', 'int', 'bool', 'string', 'union'} at 'void'

我正在使用此语法中的prebuild lexer和parser：

I'm using prebuild lexer and parser from this grammar:

grammar ProgrammingLanguage;

function_definition
    : type_specifier IDENTIFIER '(' parameter_list_opt ')' compound_statement
    ;

type_specifier
    : VOID
    | INT
    | BOOL
    | STRING
    | UNION
    ;

compound_statement
    : '{' declaration_list statement_list '}'
    ;

statement_list
    : statement
    | statement statement_list
    |
    ;

statement
    : compound_statement
    | selection_statement
    | while_statement
    | jump_statement
    | expression_statement
    | comment_statement
    ;

comment_statement
    : COMMENT_START COMMENT
    ;

selection_statement
    : IF '(' expression ')' compound_statement
    | IF '(' expression ')' compound_statement ELSE compound_statement
    | SWITCH '(' expression ')' compound_statement
    ;

expression_statement
    : ';'
    | expression ';'
    ;

jump_statement
    : BREAK ';'
    | CONTINUE ';'
    ;

while_statement
    : WHILE '(' expression ')' compound_statement
    ;

primary_expression
    : IDENTIFIER
    | CONSTANT
    | '(' expression ')'
    | IDENTIFIER '(' primary_expression_list ')'
    ;

primary_expression_list
    : primary_expression
    | primary_expression primary_expression_list
    |
    ;

expression
    : logical_or_expression
    | additive_expression
    ;

logical_or_expression
    : logical_and_expression
    | logical_or_expression '||' logical_and_expression
    ;

logical_and_expression
    : compare_expression
    | logical_and_expression '&&' compare_expression
    ;

compare_expression
    : primary_expression compare_op primary_expression
    | primary_expression
    ;

compare_op
    : '<'
    | '>'
    | '=='
    | '!='
    | '<='
    | '>='
    ;

additive_expression
    : multiplicative_expression
    | additive_expression '+' multiplicative_expression
    | additive_expression '-' multiplicative_expression
    ;

multiplicative_expression
    : primary_expression
    | multiplicative_expression '*' primary_expression
    | multiplicative_expression '/' primary_expression
    | multiplicative_expression '%' primary_expression
    ;

assignment_expression
    : IDENTIFIER '=' expression
    ;

id_list
    : IDENTIFIER
    | IDENTIFIER ',' id_list
    ;

declaration
    : type_specifier id_list ';'
    ;

parameter_list_opt
    : parameter_list
    |
    ;

parameter_list
    : type_specifier IDENTIFIER
    | type_specifier IDENTIFIER ',' parameter_list
    ;

declaration_list
    : declaration
    | declaration declaration_list
    |
    ;

/**------------------------------------------------------------------
 * LEXER RULES
 *------------------------------------------------------------------
 */
WHILE   : 'while' ;

BREAK   : 'break' ;
CONTINUE    : 'continue' ;
SWITCH  : 'switch' ;

IF  : 'if' ;
ELSE    : 'else' ;

COMMENT_START   : '//' ;

IDENTIFIER  :   ('a'..'z'|'A'..'Z')('0'..'9'|'a'..'z'|'A'..'Z')*;
CONSTANT    :   FALSE|TRUE|STRING_VALUE|INT_VALUE;
STRING_VALUE : '"'COMMENT'"';
COMMENT : ('0'..'9'|'a'..'z'|'A'..'Z')*;
INT_VALUE : ('0'..'9')+;
FALSE : 'false';
TRUE : 'true';

VOID : 'void';
INT : 'int';
BOOL : 'bool';
STRING : 'string';
UNION : 'union';

WS :    (' '|'\t'|'\n'|'\r')+ -> skip;

我正在解析这个java代码：

And I'm parsing with this java code:

import org.antlr.v4.runtime.*;
import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.ParseTreeWalker;

import java.io.IOException;

public class Main {
    public static void main(String[] args) throws IOException {
        ProgrammingLanguageLexer lexer = new ProgrammingLanguageLexer(new ANTLRFileStream("input.txt"));
        ProgrammingLanguageParser parser = new ProgrammingLanguageParser(new CommonTokenStream(lexer));
        ParseTree tree = parser.function_definition();
        ParseTreeWalker.DEFAULT.walk(new ProgrammingLanguageBaseListener(), tree);
    }
}

最后是字符串，我正在尝试解析：

And finally string, that I'm trying to parse:

void power () {}

推荐答案

错误消息表示包含值'void'的预期标记类型与通过使用字符串'void'生成的实际标记类型不匹配从输入。查看词法分析器规则表明IDENTIFIER规则正在使用输入字符串'void'，从而生成IDENTIFIER类型的标记，而不是VOID。

The error message means that the expected token type containing the value 'void' does not match the actual token type produced by consuming the string 'void' from the input. Looking at your lexer rules suggests that the input string 'void' is being consumed by the IDENTIFIER rule, producing a token of type IDENTIFIER, not VOID.

通常，匹配最长输入字符串的词法分析器获胜。对于具有相同匹配长度的两个（或更多）规则，第一个列出的胜利。将所有关键字规则移到IDENTIFIER规则之上。

In general, the lexer rule that matches longest input string wins. For two (or more) rules with the same match length, the first listed wins. Move all of your keyword rules above the IDENTIFIER rule.

有用的单元测试表将转储lex'd标记并显示匹配的实际标记类型。类似于：

A helpful unit test form will dump the lex'd tokens and show the actual token types matched. Something like:

CommonTokenStream tokens = ...
tokens.fill();
StringBuilder sb = new StringBuilder();
for (Token token : tokens.getTokens()) {
    sb.append(((YourCustomTokenType) token).toString());
}
System.out.print(sb.toString());

Token.toString（）方法通常足够好。覆盖您的令牌子类以满足您自己的需要。

The Token.toString() method is usually good enough. Override in your token subclass to fit your own needs.

这篇关于运行时期间Antlr 4.5解析器错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！