ANTLR:获取子词法分析器规则的文本表示 [英] ANTLR: Get text representation of a sub lexer rule

问题描述

考虑 ANTLR4 中的以下词法分析器规则:

ID: [a-z]+;INT:[0-9]+；数组:ID '[' INT ']';

是否有可能在我可以访问 ctx.ARRAY()(其中 ctx 是 ParserRuleContext 的子类)的树行走场景中这是从解析器规则生成的)以获得词法分析器规则 ID 和 INT 的文本表示?我目前使用 ctx.ARRAY().getText() 获取整个文本表示，并使用正则表达式解析 ID 和 INT 的内容，结果是只是想知道 ANTLR 是否提供了更清洁"的开箱即用解决方案.

注意:由于外部依赖使得 ARRAY 解析器规则不是一个选项.

预先感谢您提供有意义的答案.

解决方案

ANTLR 4 中的词法分析器规则不能分解为多个部分.这是我们做出的一个设计决策，作为 ANTLR 4 词法分析器相对于 ANTLR 3 词法分析器的大量速度和内存改进的一部分.ANTLR 3 词法分析器是递归下降识别器，具有许多与解析器相同的功能.在 ANTLR 4 中，词法分析器只不过是一个支持语义谓词的 DFA 识别器，因此根本不会跟踪标记的各个组件之间的边界.

当您需要分解标记的文本时，您必须将 ARRAY 设为解析器规则，或者单独解析 getText() 的结果.>

consider the following lexer rules in ANTLR4:

ID: [a-z]+;
INT: [0-9]+;
ARRAY: ID '[' INT ']';

Is it possible in a tree walking scenario where I have access to ctx.ARRAY() (where ctx is a subclass of ParserRuleContext that was generated out of a parser rule) to get the text representation of the lexer rules ID and INT? I currently fetch the whole text representation with ctx.ARRAY().getText() and parse the contents of ID and INT using regexes and was just wondering if there is a 'cleaner' out of the box solution ANTLR provides.

Note: Because of external dependencies making ARRAY a parser rule is not an option.

Thanks in advance for meaningful answers.

解决方案

Lexer rules in ANTLR 4 cannot be broken down into parts. This was a design decision that we made as part of a massive speed and memory improvement for ANTLR 4 lexers over ANTLR 3 lexers. ANTLR 3 lexers were recursive descent recognizers with many of the same features as parsers. In ANTLR 4, the lexer is nothing more than a DFA recognizer with support for semantic predicates, so the boundaries between the individual components of a token are not tracked at all.

You'll have to either make ARRAY a parser rule, or separately parse the result of getText() when you need to break up the token's text.

这篇关于ANTLR:获取子词法分析器规则的文本表示的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！