上下文相关的 ANTLR4 语义谓词不起作用 [英] ANTLR4 Semantic Predicates that is Context Dependent Does Not Work

问题描述

我正在用这个缩小的语法解析一个类似 C++ 的声明(删除了许多细节以使其成为一个完整的示例).它无法神秘地工作(至少对我而言).它与上下文相关谓词的使用有关吗?如果是，实现计算子节点数量逻辑"的正确方法是什么?

I am parsing a C++ like declaration with this scaled down grammar (many details removed to make it a fully working example). It fails to work mysteriously (at least to me). Is it related to the use of context dependent predicate? If yes, what is the proper way to implement the "counting the number of child nodes logic"?

grammar CPPProcessor;

cppCompilationUnit : decl_specifier_seq? init_declarator* ';'  EOF;

init_declarator:     declarator initializer?;
declarator:  identifier;
initializer: '=0';

decl_specifier_seq
  locals [int cnt=0]
    @init {  $cnt=0;    }
: decl_specifier+ ;
decl_specifier :   @init {    System.out.println($decl_specifier_seq::cnt);  }
    'const'
  | {$decl_specifier_seq::cnt < 1}? type_specifier {$decl_specifier_seq::cnt += 1;}  ;
type_specifier:  identifier ; 
identifier:IDENTIFIER;
CRLF: '\r'? '\n' -> channel(2);
WS: [ \t\f]+    -> channel(1);
IDENTIFIER:[_a-zA-Z] [0-9_a-zA-Z]* ;

我需要实现标准 C++ 规则，即 decl_specifier_seq 下允许的 type_specifier 不超过 1 个.

type_specifier 之前的语义谓词似乎是解决方案.由于嵌套的 decl_specifier_seq 是可能的，因此计数自然地被声明为 decl_specifier_seq 中的局部变量.

Semantic predicate before type_specifier seems to be the solution. And the count is naturally declared as a local variable in decl_specifier_seq since nested decl_specifier_seq are possible.

但似乎我使用的上下文相关语义谓词会产生不正确的解析，即引用 $attributes 的语义谓词.首先是一个具有正确结果的输入文件(以说明正常解析树的样子):

But it seems that a context dependent semantic predicate like the one I used will produce incorrect parsing i.e. a semantic predicate that references $attributes. First an input file with correct result (to illustrate what a normal parse tree looks like):

int t=0;

和解析树:

但是，没有'=0'的输入来帮助解析

But, an input without the '=0' to aid the parsing

int t;

0
1
line 1:4 no viable alternative at input 't'
1

解析失败，出现没有可行的替代方案"错误(控制台中打印的数字是 $decl_specifier_cnt::cnt 值的调试打印，作为测试条件的验证).即语义谓词不能阻止 t 被解析为 type_specifier 并且 t 不再被视为 init_declarator.这里有什么问题?是否因为使用了具有 $decl_specifier_seq::cnt 的上下文相关谓词?

the parsing failed with the 'no viable alternative' error (the numbers printed in the console is debug print of the $decl_specifier_cnt::cnt value as a verification of the test condition). i.e. the semantic predicate cannot prevent the t from being parsed as type_specifier and t is no longer considered a init_declarator. What is the problem here? Is it because a context dependent predicate having $decl_specifier_seq::cnt is used?

这是否意味着上下文相关谓词不能用于实现计算子节点数量"的逻辑?

Does it mean context dependent predicate cannot be used to implement "counting the number of child nodes" logic?

编辑

我尝试了谓词使用成员变量而不是 $decl_specifier_seq::cnt 的新版本，令人惊讶的是，语法现在可以证明上下文相关谓词确实导致以前的语法失败:

I tried new versions whose predicate uses member variable instead of the $decl_specifier_seq::cnt and surprisingly the grammar now works proving that the Context Dependent predicate did cause the previous grammar to fail:

....
@parser::members {
  public int cnt=0;
}
decl_specifier
  @init {System.out.println("cnt:"+cnt);  }
: 
    'const'
  | {cnt<1 }? type_specifier {cnt++;}  ;

产生一个正常的解析树:

A normal parse tree is resulted:

这就产生了一个问题，如果我们必须使用成员变量来代替局部变量以避免上下文敏感谓词，那么如何支持嵌套规则?

This gives rise to the question of how to support nested rule if we must use member variables to replace the local variables to avoid context sensitive predicates?

一个奇怪的结果是，如果我在谓词后添加一个 /*$ctx*/ ，它又失败了:

And a weird result is that if I add a /*$ctx*/ after the predicate, it fails again:

decl_specifier
  @init {System.out.println("cnt:"+cnt);  }
: 
    'const'
  | {cnt<1 /*$ctx*/ }? type_specifier {cnt++;}  ;

line 1:4 no viable alternative at input 't'

解析失败，没有可行的替代方案.为什么 /*$ctx*/ 会像使用 $decl_specifier_seq::cnt 一样导致解析失败，尽管实际逻辑仅使用成员变量?而且，如果没有 /*$ctx*/，另一个与 @init 块之前调用的谓词相关的问题出现(

The parsing failed with no viable alternative. Why the /*$ctx*/ causes the parsing to fail like when $decl_specifier_seq::cnt is used although the actual logic uses a member variable only? And, without the /*$ctx*/, another issue related to the predicate called before @init block appears(described here)

推荐答案

ANTLR 4 在两种情况下评估语义谓词.

ANTLR 4 evaluates semantic predicates in two cases.

1. 生成的代码在解析过程中评估语义谓词，并抛出评估返回 false 的异常.解析过程中遍历的所有谓词都以这种方式进行评估，包括上下文相关谓词和未出现在决策左侧的谓词.
2. 预测方法会评估谓词，以便在解析过程中做出正确的决定.在这种情况下，假定出现在被评估决策左边缘以外的任何地方的谓词返回真(即它们被忽略).此外，仅当上下文数据可用时才评估上下文相关谓词.预测算法不会创建解析代码尚未提供的上下文结构.如果在预测过程中遇到依赖于上下文的谓词并且没有可用的上下文，则假定该谓词返回 true(即，该决策将忽略该谓词).

代码生成器不评估目标语言的语义，因此无法知道$ctx出现在/*$ctx*/<时是语义无关的/代码>.这两种情况都会导致谓词被视为依赖于上下文.

The code generator does not evaluate the semantics of the target language, so it has no way to know that $ctx is semantically irrelevant when it appears in /*$ctx*/. Both cases result in the predicate being treated as context-dependent.

这篇关于上下文相关的 ANTLR4 语义谓词不起作用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！