有没有使用antlr4从Java源代码创建AST并提取方法，变量和注释的简单示例? [英] Is there a simple example of using antlr4 to create an AST from java source code and extract methods, variables and comments?

问题描述

有人可以提供有关我如何使用antlr4进行此操作的详细示例吗?正确安装antlr4及其依赖项的说明将不胜感激.

Can someone provide a detailed example as to how I can do this using antlr4? Instructions right from installing antlr4 and its dependencies would be highly appreciated.

推荐答案

在这里.

首先，您要购买ANTLR4书;-)

First, you're gonna buy the ANTLR4 book ;-)

第二，您将下载antlr4 jar和Java语法( http ://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference )

Second, you'll download antlr4 jar and the java grammar (http://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference)

然后，您可以对语法进行一些更改，将其添加到标题中

Then, you can change the grammar a little bit, adding these to the header

    (...)
grammar Java;

options 
{
    language = Java;
}

// starting point for parsing a java file
compilationUnit
    (...)

我将在语法上做一些改动，以说明问题.

I'll change a little thing in the grammar just to illustrate something.

/*
methodDeclaration
    :   (type|'void') Identifier formalParameters ('[' ']')*
        ('throws' qualifiedNameList)?
        (   methodBody
        |   ';'
        )
    ;
*/
methodDeclaration
    :   (type|'void') myMethodName formalParameters ('[' ']')*
        ('throws' qualifiedNameList)?
        (   methodBody
        |   ';'
        )
    ;

myMethodName
    :   Identifier
    ;

您会看到，原始语法不允许您从任何其他标识符中识别方法标识符，因此我对原始块进行了注释，并添加了一个新块，目的只是向您展示如何获得所需的内容.

You see, the original grammar does not let you identify the method identifier from any other identifier, so I've commented the original block and added a new one just to show you how to get what you want.

您必须对要检索的其他元素(例如注释)执行相同的操作，这些元素当前仅被跳过.那是给你的:-)

You'll have to do the same for other elements you want to retrieve, like the comments, that are currently being just skipped. That's for you :-)

现在，创建一个像这样的类以生成所有存根

Now, create a class like this to generate all the stubs

package mypackage;

public class Gen {

    public static void main(String[] args) {
        String[] arg0 = { "-visitor", "/home/leoks/EclipseIndigo/workspace2/SO/src/mypackage/Java.g4", "-package", "mypackage" };
        org.antlr.v4.Tool.main(arg0);
    }

}

运行Gen，您将在mypackage中获得一些为您创建的Java代码.

Run Gen, and you'll get some java code created for you in mypackage.

现在创建一个访问者.实际上，在此示例中，访问者将自行解析

Now create a Visitor. Actually, the visitor will parse itself in this example

package mypackage;

import java.io.FileInputStream;
import java.io.IOException;

import mypackage.JavaParser.MyMethodNameContext;

import org.antlr.v4.runtime.ANTLRInputStream;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.ParseTreeWalker;

/**
 * @author Leonardo Kenji Feb 4, 2014
 */
public class MyVisitor extends JavaBaseVisitor<Void> {

    /**
     * Main Method
     * 
     * @param args
     * @throws IOException
     */
    public static void main(String[] args) throws IOException {
        ANTLRInputStream input = new ANTLRInputStream(new FileInputStream("/home/leoks/EclipseIndigo/workspace2/SO/src/mypackage/MyVisitor.java")); // we'll
                                                                                                                                                    // parse
                                                                                                                                                    // this
                                                                                                                                                    // file
        JavaLexer lexer = new JavaLexer(input);
        CommonTokenStream tokens = new CommonTokenStream(lexer);
        JavaParser parser = new JavaParser(tokens);
        ParseTree tree = parser.compilationUnit(); // see the grammar ->
                                                    // starting point for
                                                    // parsing a java file



        MyVisitor visitor = new MyVisitor(); // extends JavaBaseVisitor<Void>
                                                // and overrides the methods
                                                // you're interested
        visitor.visit(tree);
    }

    /**
     * some attribute comment
     */
    private String  someAttribute;

    @Override
    public Void visitMyMethodName(MyMethodNameContext ctx) {
        System.out.println("Method name:" + ctx.getText());
        return super.visitMyMethodName(ctx);
    }

}

就是这样.

您会得到类似的东西

Method name:main
Method name:visitMyMethodName

ps.还有一件事.当我用eclipse编写此代码时，我遇到了一个奇怪的异常.这是由Java 7引起的，只需将这些参数添加到编译器中即可解决(由于此链接 http ://java.dzone.com/articles/javalangverifyerror-expecting )

ps. one more thing. While I was writing this code in eclipse, I've got a strange exception. This is caused by Java 7 and can be fixed just adding these parameters to your compiler (thanks to this link http://java.dzone.com/articles/javalangverifyerror-expecting)

这篇关于有没有使用antlr4从Java源代码创建AST并提取方法，变量和注释的简单示例?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！