是“解析器规则中的隐式令牌定义"吗?有什么好担心的? [英] Is "Implicit token definition in parser rule" something to worry about?

问题描述

我正在用 ANTLR 和 ANTLRWorks 2 创建我的第一个语法.我基本上已经完成了语法本身(它识别用所描述的语言编写的代码并构建正确的解析树)，但除此之外我还没有开始任何事情.

I'm creating my first grammar with ANTLR and ANTLRWorks 2. I have mostly finished the grammar itself (it recognizes the code written in the described language and builds correct parse trees), but I haven't started anything beyond that.

让我担心的是，解析器规则中每个第一次出现的标记都带有黄色波浪线下划线，上面写着解析器规则中的隐式标记定义".

What worries me is that every first occurrence of a token in a parser rule is underlined with a yellow squiggle saying "Implicit token definition in parser rule".

例如，在此规则中，'var' 具有波浪线:

For example, in this rule, the 'var' has that squiggle:

variableDeclaration: 'var' IDENTIFIER ('=' expression)?;

它的样子:

奇怪的是，ANTLR 本身似乎并不介意这些规则(在进行测试设备测试时，我在解析器生成器输出中看不到任何这些警告，只是在我的计算机上安装了不正确的 Java 版本机)，所以这只是 ANTLRWorks 的抱怨.

The odd thing is that ANTLR itself doesn't seem to mind these rules (when doing test rig test, I can't see any of these warning in the parser generator output, just something about incorrect Java version being installed on my machine), so it's just ANTLRWorks complaining.

是否需要担心或忽略这些警告?我应该在词法分析器规则中明确声明所有标记吗? 官方圣经中的大多数示例 权威 ANTLR 参考 似乎完全按照我编写代码的方式完成.

Is it something to worry about or should I ignore these warnings? Should I declare all the tokens explicitly in lexer rules? Most exaples in the official bible The Defintive ANTLR Reference seem to be done exactly the way I write the code.

推荐答案

我强烈建议在任何重要的代码中更正此警告的所有实例.

此警告是(实际上是由我创建的)是为了提醒您注意以下情况:

This warning was created (by me actually) to alert you to situations like the following:

shiftExpr : ID (('<<' | '>>') ID)?;

由于 ANTLR 4 鼓励将动作代码编写在目标语言中的单独文件中，而不是将它们直接嵌入到语法中，因此能够区分 << 和 很重要>>.如果没有为这些运算符明确创建令牌，它们将被分配任意类型，并且没有命名常量可用于引用它们.

Since ANTLR 4 encourages action code be written in separate files in the target language instead of embedding them directly in the grammar, it's important to be able to distinguish between << and >>. If tokens were not explicitly created for these operators, they will be assigned arbitrary types and no named constants will be available for referencing them.

此警告还有助于避免以下有问题的情况:

This warning also helps avoid the following problematic situations:

• 解析器规则包含拼写错误的标记引用.如果没有警告，这可能会导致静默创建可能永远不会匹配的额外令牌.

• 解析器规则包含无意的标记引用，例如:

• A parser rule contains a misspelled token reference. Without the warning, this could lead to silent creation of an additional token that may never be matched.

• A parser rule contains an unintentional token reference, such as the following:

number : zero | INTEGER;
zero   : '0'; // <-- this implicit definition causes 0 to get its own token

这篇关于是“解析器规则中的隐式令牌定义"吗?有什么好担心的?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！