使用 Antlr 获取标识符和函数名称 [英] Using Antlr to get identifiers and function names

问题描述

我正在尝试使用和理解 AntLR，这对我来说是新的.我的目的是读取用 C 编写的源代码文件并从中提取标识符(变量和函数名称).

I'm trying to use and understand AntLR, this is new to me. My purpose is to read a source code file written in C and extract from it the identifiers (variables and function names).

在我的 C 语法(文件 C.g4)中考虑:

In my C grammar (file C.g4) consider:

identifierList
    :   Identifier
    |   identifierList Comma Identifier
    ;
Identifier
    :   IdentifierNondigit
        (   IdentifierNondigit
        |   Digit
        )*
    ;

在生成解析器和侦听器后，我为 identifierList 创建了自己的侦听器.

After generation of parser and listener I create my own listener to the identifierList.

请注意，MyCListener 类扩展了 CBaseListener:

Note that MyCListener class extends CBaseListener:

public class MyCListener extends CBaseListener {


@Override
public void enterIdentifierList(CParser.IdentifierListContext ctx) {
    List<ParseTree> children = ctx.children;
    for (ParseTree parseTree : children) {
        System.out.println(parseTree.getText());
    }

}

然后我在主课上有这个:

Then I have this in main class:

 String fileurl = "C:/example.c";

 CLexer lexer;
 try {
       lexer = new CLexer(new ANTLRFileStream(fileurl));
       CommonTokenStream tokens = new CommonTokenStream(lexer);
       CParser parser = new CParser(tokens);

       CParser.IdentifierListContext identifierContext = parser.identifierList();
       ParseTreeWalker walker = new ParseTreeWalker();
       MyCListener listener = new MyCListener();
       walker.walk(listener, identifierContext);

 } catch (IOException ex) {
       Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
 }

example.c 在哪里:

Where example.c is:

int main() {

// this is C

 int i=0; // i is int
 /* double j=0.0;
    C
 */
}

我做错了什么?可能是我没有写好MyCListener，或者identifierList不是我需要听的……真的不知道.抱歉，我什至不明白我的输出，为什么会出现词法错误?:

What am I doing wrong? Maybe I didn't write MyCListener properly, or identifierList is not what I need to listen... Really don't know. I'm sorry, but I didn't even understand my output, why is there a lexical error?:

line 3:4 mismatched input '(' expecting {<EOF>, ','}
main
(
)
{
int
i
=
0
;
}

如您所见，我对此感到非常困惑.有人可以帮助我吗?请...

As you see, I'm very confused about this. Can somebody help me ? Please...

推荐答案

用这一行:

CParser.IdentifierListContext identifierContext = parser.identifierList();

您正在尝试将整个输入解析为 identifierList.但您的意见不仅如此.

you're trying to parse your entire input as an identifierList. But your input isn't just that.

假设您使用的是 C.g4来自ANTLR4 Github存储库，尝试让解析器从语法的入口点开始(这是规则compilationUnit):

Assuming you're using the C.g4 from the ANTLR4 Github repository, try to let the parser start at the entry point of the grammar (which is the rule compilationUnit):

MyCListener listener = new MyCListener();
ParseTreeWalker.DEFAULT.walk(listener, parser.compilationUnit());

编辑

这是一个快速演示:

EDIT

Here's a quick demo:

public class Main {

    public static void main(String[] args) throws Exception {

        final List<String> identifiers = new ArrayList<String>();

        String source = "int main() {\n" +
                "\n" +
                "// this is C\n" +
                "\n" +
                " int i=0; // i is int\n" +
                " /* double j=0.0;\n" +
                "    C\n" +
                " */\n" +
                "}";

        CLexer lexer = new CLexer(new ANTLRInputStream(source));
        CParser parser = new CParser(new CommonTokenStream(lexer));

        ParseTreeWalker.DEFAULT.walk(new CBaseListener(){

            @Override
            public void enterDirectDeclarator(@NotNull CParser.DirectDeclaratorContext ctx) {
                if (ctx.Identifier() != null) {
                    identifiers.add(ctx.Identifier().getText());
                }
            }

            // Perhaps override other rules that use `Identifier`

        }, parser.compilationUnit());

        System.out.println("identifiers -> " + identifiers);
    }
}

将打印:

identifiers -> [main, i]

这篇关于使用 Antlr 获取标识符和函数名称的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！