ANTLR-带空格的标识符 [英] ANTLR - identifier with whitespace

问题描述

我想要可以包含空格的标识符.

i want identifiers that can contain whitespace.

grammar WhitespaceInSymbols;

premise :   ( options {greedy=false;} : 'IF' )  id=ID{
System.out.println($id.text);
};

ID  :   ('a'..'z'|'A'..'Z')+ (' '('a'..'z'|'A'..'Z')+)* 
;

WS  :   ' '+ {skip();}
;

当我使用分析的IF语句"对此进行测试时，我得到了MissingTokenException和输出分析的IF语句".
我以为，通过使用greedy = false，我可以告诉ANTLR在'IF'之后退出并将其作为令牌.但是，IF是ID的一部分. 有没有办法实现我的目标?我已经尝试过greed = false-option的一些变体，但没有成功.

When i test this with "IF statement analyzed" i get a MissingTokenException and the output "IF statement analyzed".
I thought, that by using greedy=false i could tell ANTLR to exit afer 'IF' and take it as a token. But instead the IF is part of the ID. Is there a way to achieve my goal? I already tried some variations of the greed=false-option, but without success.

推荐答案

我认为，通过使用greedy = false，我可以告诉ANTLR退出"IF"并将其作为令牌.

I thought, that by using greedy=false i could tell ANTLR to exit afer 'IF' and take it as a token.

否，解析器对令牌的创建没有什么可说的:首先对输入进行令牌化，然后将解析器规则应用于这些令牌.因此设置greedy=false无效.

No, the parser has nothing to say about the creation of tokens: the input is first tokenized and then the parser rules are applied on these tokens. So setting greedy=false has no effect.

您可以 做到这一点(创建带有空白的ID令牌)，但这将是一个可怕的解决方案，其中包含许多谓词，而lexer中的一些自定义方法也可以进行手动前瞻:您真的，真的不想要这个！一种更简洁的解决方案是在解析器中引入id规则，并使其与一个或多个ID令牌匹配.

You can do this (creating ID tokens with white spaces), but it will be a horrible solution with many predicates, and a few custom methods in the lexer doing manual look-aheads: you really, really don't want this! A much cleaner solution would be to introduce a id rule in your parser and let it match one or more ID tokens.

grammar WhitespaceInSymbols;

premise
  :  IF id THEN EOF
  ;

id
  :  ID+
  ;

IF
  :  'IF'
  ;

THEN
  :  'THEN'
  ;

ID  
  :  ('a'..'z' | 'A'..'Z')+
  ;

WS  
  :  ' '+ {skip();}
  ;

会将输入IF statement analyzed THEN解析到以下树中:

would parse the input IF statement analyzed THEN into the following tree:

这篇关于ANTLR-带空格的标识符的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！