是否有使用 antlr4 从 java 源代码创建 AST 并提取方法、变量和注释的简单示例? [英] Is there a simple example of using antlr4 to create an AST from java source code and extract methods, variables and comments?

问题描述

有人可以提供一个详细的例子来说明我如何使用 antlr4 做到这一点吗?非常感谢安装 antlr4 及其依赖项的说明.

Can someone provide a detailed example as to how I can do this using antlr4? Instructions right from installing antlr4 and its dependencies would be highly appreciated.

推荐答案

在这里.

首先，您要购买 ANTLR4 书 ;-)

First, you're gonna buy the ANTLR4 book ;-)

其次，您将下载 antlr4 jar 和 java 语法(http://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference)

Second, you'll download antlr4 jar and the java grammar (http://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference)

然后，您可以稍微更改语法，将这些添加到标题中

Then, you can change the grammar a little bit, adding these to the header

    (...)
grammar Java;

options 
{
    language = Java;
}

// starting point for parsing a java file
compilationUnit
    (...)

为了说明一些东西，我会在语法上做一些改动.

I'll change a little thing in the grammar just to illustrate something.

/*
methodDeclaration
    :   (type|'void') Identifier formalParameters ('[' ']')*
        ('throws' qualifiedNameList)?
        (   methodBody
        |   ';'
        )
    ;
*/
methodDeclaration
    :   (type|'void') myMethodName formalParameters ('[' ']')*
        ('throws' qualifiedNameList)?
        (   methodBody
        |   ';'
        )
    ;

myMethodName
    :   Identifier
    ;

您看，原始语法不允许您从任何其他标识符中识别方法标识符，因此我对原始块进行了注释并添加了一个新块，只是为了向您展示如何获得您想要的.

You see, the original grammar does not let you identify the method identifier from any other identifier, so I've commented the original block and added a new one just to show you how to get what you want.

您必须对要检索的其他元素(例如当前刚刚跳过的评论)执行相同操作.那是给你的:-)

You'll have to do the same for other elements you want to retrieve, like the comments, that are currently being just skipped. That's for you :-)

现在，创建一个这样的类来生成所有存根

Now, create a class like this to generate all the stubs

package mypackage;

public class Gen {

    public static void main(String[] args) {
        String[] arg0 = { "-visitor", "/home/leoks/EclipseIndigo/workspace2/SO/src/mypackage/Java.g4", "-package", "mypackage" };
        org.antlr.v4.Tool.main(arg0);
    }

}

运行 Gen，您将在 mypackage 中获得一些为您创建的 Java 代码.

Run Gen, and you'll get some java code created for you in mypackage.

现在创建一个访问者.实际上，在这个例子中，访问者会解析自己

Now create a Visitor. Actually, the visitor will parse itself in this example

package mypackage;

import java.io.FileInputStream;
import java.io.IOException;

import mypackage.JavaParser.MyMethodNameContext;

import org.antlr.v4.runtime.ANTLRInputStream;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.ParseTreeWalker;

/**
 * @author Leonardo Kenji Feb 4, 2014
 */
public class MyVisitor extends JavaBaseVisitor<Void> {

    /**
     * Main Method
     * 
     * @param args
     * @throws IOException
     */
    public static void main(String[] args) throws IOException {
        ANTLRInputStream input = new ANTLRInputStream(new FileInputStream("/home/leoks/EclipseIndigo/workspace2/SO/src/mypackage/MyVisitor.java")); // we'll
                                                                                                                                                    // parse
                                                                                                                                                    // this
                                                                                                                                                    // file
        JavaLexer lexer = new JavaLexer(input);
        CommonTokenStream tokens = new CommonTokenStream(lexer);
        JavaParser parser = new JavaParser(tokens);
        ParseTree tree = parser.compilationUnit(); // see the grammar ->
                                                    // starting point for
                                                    // parsing a java file



        MyVisitor visitor = new MyVisitor(); // extends JavaBaseVisitor<Void>
                                                // and overrides the methods
                                                // you're interested
        visitor.visit(tree);
    }

    /**
     * some attribute comment
     */
    private String  someAttribute;

    @Override
    public Void visitMyMethodName(MyMethodNameContext ctx) {
        System.out.println("Method name:" + ctx.getText());
        return super.visitMyMethodName(ctx);
    }

}

就是这样.

你会得到类似的东西

Method name:main
Method name:visitMyMethodName

ps.还有一件事.当我在 eclipse 中编写这段代码时，我遇到了一个奇怪的异常.这是由 Java 7 引起的，只需将这些参数添加到编译器即可修复(感谢此链接 http://java.dzone.com/articles/javalangverifyerror-expecting)

ps. one more thing. While I was writing this code in eclipse, I've got a strange exception. This is caused by Java 7 and can be fixed just adding these parameters to your compiler (thanks to this link http://java.dzone.com/articles/javalangverifyerror-expecting)

这篇关于是否有使用 antlr4 从 java 源代码创建 AST 并提取方法、变量和注释的简单示例?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！