ANTLR:获取子词法分析器规则的文本表示形式 [英] ANTLR: Get text representation of a sub lexer rule

问题描述

在ANTLR4中考虑以下词法分析器规则:

consider the following lexer rules in ANTLR4:

ID: [a-z]+;
INT: [0-9]+;
ARRAY: ID '[' INT ']';

在可以访问ctx.ARRAY()(其中ctx是解析器规则生成的ParserRuleContext的子类)的树行走场景中，是否有可能获取词法分析器规则的文本表示ID和INT? 我目前使用ctx.ARRAY().getText()获取整个文本表示形式，并使用正则表达式解析ID和INT的内容，只是想知道ANTLR是否提供开箱即用的清洁器"解决方案.

Is it possible in a tree walking scenario where I have access to ctx.ARRAY() (where ctx is a subclass of ParserRuleContext that was generated out of a parser rule) to get the text representation of the lexer rules ID and INT? I currently fetch the whole text representation with ctx.ARRAY().getText() and parse the contents of ID and INT using regexes and was just wondering if there is a 'cleaner' out of the box solution ANTLR provides.

注意:由于外部依赖性，使得ARRAY解析器规则不可行.

Note: Because of external dependencies making ARRAY a parser rule is not an option.

预先感谢您提供有意义的答案.

Thanks in advance for meaningful answers.

推荐答案

ANTLR 4中的Lexer规则不能分解为多个部分.这是我们作为ANTLR 4 lexers相对于ANTLR 3 lexers大幅提高速度和内存的一部分而做出的设计决策. ANTLR 3词法分析器是递归血统识别器，具有许多与解析器相同的功能.在ANTLR 4中，词法分析器不过是支持语义谓词的DFA识别器，因此根本不会跟踪令牌的各个组成部分之间的边界.

Lexer rules in ANTLR 4 cannot be broken down into parts. This was a design decision that we made as part of a massive speed and memory improvement for ANTLR 4 lexers over ANTLR 3 lexers. ANTLR 3 lexers were recursive descent recognizers with many of the same features as parsers. In ANTLR 4, the lexer is nothing more than a DFA recognizer with support for semantic predicates, so the boundaries between the individual components of a token are not tracked at all.

您需要制作ARRAY解析器规则，或者在需要拆分令牌文本时单独解析getText()的结果.

You'll have to either make ARRAY a parser rule, or separately parse the result of getText() when you need to break up the token's text.

这篇关于ANTLR:获取子词法分析器规则的文本表示形式的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！