ANTLR java测试文件无法创建树语法对象 [英] ANTLR java test file can't create object of tree grammar

问题描述

我正在使用针对 java 的 ANTLR 3.x 创建解析器.我已经编写了解析器语法(用于创建抽象语法树，AST)和树语法(用于在 AST 上执行操作).最后，为了测试这两个语法文件，我用 Java 编写了一个测试文件.

I am creating a parser using ANTLR 3.x that targets java. I have written both parser grammar (for creating Abstract Syntax Tree, AST) and Tree Grammar (for performing operations on AST). Finally, to test both grammar files, I have written a test file in Java.

看看下面的代码，

协议语法

grammar protocol;
options {
      language = Java;
  output = AST;
}

tokens{ //imaginary tokens
PROT;
INITIALP;
PROC;
TRANSITIONS;
}
@header {
import twoprocess.Configuration;
package com.javadude.antlr3.x.tutorial;
}

@lexer::header {
  package com.javadude.antlr3.x.tutorial;
}
/*
parser rules, in lowercase letters
*/
program
    : declaration+
    ;
declaration
    :protocol
    |initialprocess
    |process
    |transitions
    ;

protocol
    :'protocol' ID ';' -> ^(PROT ID)
    ;
initialprocess
    :'pin' '=' INT ';' -> ^(INITIALP INT)
    ;
process
    :'p' '=' INT ';' -> ^(PROC INT)
    ;
transitions
    :'transitions' '=' INT ('(' INT ',' INT ')') + ';' -> ^(TRANSITIONS INT INT INT*)
    ;

/*
lexer rules (tokens), in upper case letters
*/
ID  
    : (('a'..'z' | 'A'..'Z'|'_')('a'..'z' | 'A'..'Z'|'0'..'9'|'_'))*;
INT 
    : ('0'..'9')+;
WHITESPACE
    : ('\t' | ' ' | '\r' | '\n' | '\u000C')+ {$channel = HIDDEN;};

protocolWalker

grammar protocolWalker;

options {
  language = Java;
  //Error, eclipse can't access tokenVocab named protocol
  tokenVocab = protocol;    //import tokens from protocol.g i.e, from protocol.tokens file
  ASTLabelType = CommonTree;
  }

@header {
import twoprocess.Configuration;
package com.javadude.antlr3.x.tutorial;
}

program
    : declaration+
    ;

declaration
    :protocol
    |initialprocess
    |process
    |transitions
    ;

protocol
    :^(PROT ID)
    {System.out.println("create protocol " +$ID.text);}
    ;

initialprocess
    :^(INITIALP INT)
    {System.out.println("");}
    ;

process
    :^(PROC INT)
    {System.out.println("");}
    ;

transitions
    :^(TRANSITIONS INT INT INT*)
    {System.out.println("");}
    ;

Protocoltest.java

package com.javadude.antlr3.x.tutorial;  
import org.antlr.runtime.*;  
import org.antlr.runtime.tree.*;  
import org.antlr.runtime.tree.CommonTree;  
import org.antlr.runtime.tree.CommonTreeNodeStream;  
public class Protocoltest { 



/**
 * @param args
 */
public static void main(String[] args) throws Exception {
    //create input stream from standard input
    ANTLRInputStream input = new ANTLRInputStream(System.in);
    //create a lexer attached to that input stream
    protocolLexer lexer = new protocolLexer(input);
    //create a stream of tokens pulled from the lexer
    CommonTokenStream tokens = new CommonTokenStream(lexer);

    //create a pareser attached to teh token stream
    protocolParser parser = new protocolParser(tokens);
    //invoke the program rule in get return value
    protocolParser.program_return r =parser.program();
    CommonTree t = (CommonTree)r.getTree();
    //output the extracted tree to the console
    System.out.println(t.toStringTree());

    //walk resulting tree; create treenode stream first
    CommonTreeNodeStream nodes = new CommonTreeNodeStream(t);
    //AST nodes have payloads that point into token stream
    nodes.setTokenStream(tokens);


    //create a tree walker attached to the nodes stream  
            //Error, can't create TreeGrammar object called walker
    protocolWalker walker = new protocolWalker(nodes);

    //invoke the start symbol, rule program
    walker.program();
    }
}

问题:

1. 在 protocolWalker 中，我无法访问令牌 (protocol.tokens)

1. In protocolWalker, I can't access the tokens (protocol.tokens)

//Error, eclipse can't access tokenVocab named protocol  
    tokenVocab = protocol; //import tokens from protocol.g i.e, from protocol.tokens file

在protocolWalker中，能否在action列表中创建java类的对象，名为Configuration?

In In protocolWalker, can I create the object of java class, called Configuration, in the action list?

protocol
    :^(PROT ID)
       {System.out.println("create protocol " +$ID.text);
        Configuration conf = new Configuration();
       }
    ;

在 Protocoltest.java 中

In Protocoltest.java

//create a tree walker attached to the nodes stream    
//Error, can't create TreeGrammar object called walker  
    protocolWalker walker = new protocolWalker(nodes);  

无法创建 protocolWalker 的对象.我在示例和教程中看到创建了这样的对象.

Object of protocolWalker can't be created. I have seen in the examples and the tutorials that such object is created.

推荐答案

在 protocolWalker 中，我无法访问令牌 (protocol.tokens)...

In protocolWalker, I can't access the tokens (protocol.tokens)...

似乎可以很好地访问 protocol.tokens:将 tokenVocab 更改为其他内容会产生现在不会产生的错误.protocolWalker.g 的问题在于它被定义为标记解析器(grammar protocolWalker)，但它被用作树解析器.将语法定义为 treegrammar protocolWalker 消除了我看到的有关未定义标记的错误.

It seems to be accessing protocol.tokens fine: changing tokenVocab to something else produces an error that it doesn't produce now. The problem with protocolWalker.g is that it's defined as a token parser (grammar protocolWalker) but it's being used like a tree parser. Defining the grammar as tree grammar protocolWalker took away the errors that I was seeing about the undefined tokens.

在protocolWalker中，是否可以在action列表中创建java类的对象，名为Configuration?

In protocolWalker, can I create the object of java class, called Configuration, in the action list?

是的，你可以.普通的 Java 编程警告适用于导入类等，但它与 System.out.println 之类的代码一样可供您使用.

Yes, you can. The normal Java programming caveats apply about importing the class and so on, but it's as available to you as code like System.out.println.

在Protocoltest.java ... 无法创建protocolWalker 的对象.

In Protocoltest.java ... Object of protocolWalker can't be created.

protocolWalker.g(就像现在一样)生成一个名为 protocolWalkerParser 的标记解析器.当您将其更改为树语法时，它将生成一个名为 protocolWalker 的树解析器.

protocolWalker.g (as it is now) produces a token parser named protocolWalkerParser. When you change it to a tree grammar, it'll produce a tree parser named protocolWalker instead.

非常感谢您发布整个语法.这让回答问题变得容易多了.

Thanks a lot for posting the whole grammars. That made answering the question much easier.

这篇关于ANTLR java测试文件无法创建树语法对象的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！