ANTLR 语法迁移工具 [英] Migration tool for ANTLR grammar

问题描述

假设我有以下简单的语法(查询 DSL):

Suppose I have a following simple grammar (query DSL):

grammar TestGrammar;

term : textTerm ;

textTerm : 'Text' '(' T_VALUE '=' STRING+ ')' ;

T_VALUE : 'value' ;
STRING : '"' .+? '"' ;

WS : [ \t\r\n]+ -> skip ;

然后在某些时候我决定需要更改文本术语格式，例如:

Then at some point I decide that text term format needs to be changed, for example:

Text(value = "123") -> MyText(val = "123")

我应该如何迁移用户使用以前版本的语法生成的现有数据?

How should I approach migrating existing data that users have generated with previous version of grammar?

推荐答案

假设

让我们通过为 'Text' 字符串引入标记 TEXT 来简化您的语法.

Assumption

Let's make one simplification of your grammar by introducing token TEXT for 'Text' string.

grammar TestGrammar;

WS : [ \t\r\n]+ -> channel(HIDDEN);  // preserve the whitespaces characters!
T_VALUE : 'value';
STRING : '"' .+? '"';
TEXT : 'Text';

term
    : textTerm;
textTerm
    : TEXT '(' T_VALUE '=' STRING+ ')';

解决方案

现在我们将使用由 ANTLRv4 工具构建的 AST 侦听器.这允许我们遍历 AST 并使用 Lucas Trzesniewski 在评论中提到的 TokenStreamRewriter 类执行令牌替换.

Solution

Now we will utilize the AST listener built by ANTLRv4 tool. This allows us to traverse AST and perform token replacement with TokenStreamRewriter class already mentioned by Lucas Trzesniewski in comments.

import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.TokenStreamRewriter;

public class MigrationTask extends TestGrammarBaseListener {
    private TokenStreamRewriter rewriter;

    public MigrationTask(CommonTokenStream stream) {
        this.rewriter = new TokenStreamRewriter(stream);
    }

    @Override
    public void enterTextTerm(TestGrammarParser.TextTermContext ctx) {
        rewriter.replace(ctx.TEXT().getSymbol(), "MyText");
        rewriter.replace(ctx.T_VALUE().getSymbol(), "val");
    }

    public String getMigrationResult() {
        return rewriter.getText();
    }
}

因此，每当我们在遍历 AST 过程中遇到令牌时，我们就用它的替代物替换给定的令牌.

So, we substitute given token with its replacement whenever we encounter the token during the traversal of AST.

现在我们可以在给定的 ParseTree 上执行 MigrationTask 并检索迁移结果:

Now we can execute MigrationTask on given ParseTree and retrive the migration result:

(...)
CommonTokenStream tokens = new CommonTokenStream(lexer);
TestGrammarParser parser = new TestGrammarParser(tokens);
ParseTree tree = parser.term();
ParseTreeWalker walker = new ParseTreeWalker();
MigrationTask migrationTask = new MigrationTask(tokens);
walker.walk(migrationTask, tree);
String result = migrationTask.getMigrationResult(); // Here we retrive migration result !
(...)

这篇关于ANTLR 语法迁移工具的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！