替换 ANTLR 中的令牌 [英] Replace token in ANTLR
问题描述
我想使用 ANTLR 替换令牌.
我尝试使用 TokenRewriteStream 并替换,但没有奏效.
有什么建议吗?
ANTLRStringStream in = new ANTLRStringStream(source);MyLexer 词法分析器 = new MyLexer(in);TokenRewriteStream 令牌 = new TokenRewriteStream(lexer);for(Object obj : tokens.getTokens()) {CommonToken token = (CommonToken)obj;令牌.替换(令牌,替换");}
词法分析器找到所有出现的单行注释,我也想在原始源中替换它们.
这是语法:
语法ANTLRTest;选项 {语言 = Java;}@header {包主;}@lexer::header {包主;}规则:SingleLineComment+;单行注释: '//' ~( '\r' | '\n' )* {setText("replacement");};
我想做的是替换文件中的所有单行注释,比方说.
重写词法分析器中的文本:
SingleLineComment: '//' ~('\r' | '\n')* {setText("replacement");};
编辑
好的,这是一个如何从语言中过滤某些标记的快速演示:
SingleCommentStrip.g
grammar SingleCommentStrip;解析返回 [String str]@init{StringBuilder builder = new StringBuilder();}:(t=.{builder.append($t.text);})* EOF {$str = builder.toString();};单行注释: '//' ~('\r' | '\n')* {skip();};多行注释: '/*' .* '*/';字符串字面量: '"' ('\\' . | ~('"' | '\\' | '\r' | '\n'))* '"';任何其他字符:.;
Main.java
import org.antlr.runtime.*;公共课主要{public static void main(String[] args) 抛出异常 {SingleCommentStripLexer 词法分析器 = new SingleCommentStripLexer(new ANTLRFileStream("Test.java"));SingleCommentStripParser parser = new SingleCommentStripParser(new CommonTokenStream(lexer));字符串调整 = parser.parse();System.out.println(已调整);}}
测试.java
//评论类测试{/*//不删除*///也评论String s = "/* don't//remove */\" \\ me */as well";}
现在运行演示:
java -cp antlr-3.3.jar org.antlr.Tool SingleCommentStrip.gjavac -cp antlr-3.3.jar *.javajava -cp .:antlr-3.3.jar Main
将打印:
<前>类测试{/*//不删除*/String s = "/* don't//remove */\" \\ me */as well";}(即删除单行注释)
I want to replace a token using ANTLR.
I tried with TokenRewriteStream and replace, but it didn't work.
Any suggestions?
ANTLRStringStream in = new ANTLRStringStream(source);
MyLexer lexer = new MyLexer(in);
TokenRewriteStream tokens = new TokenRewriteStream(lexer);
for(Object obj : tokens.getTokens()) {
CommonToken token = (CommonToken)obj;
tokens.replace(token, "replacement");
}
The lexer finds all occurences of single-line comments, and i want to replace them in the original source too.
EDIT:
This is the grammar:
grammar ANTLRTest;
options {
language = Java;
}
@header {
package main;
}
@lexer::header {
package main;
}
rule: SingleLineComment+;
SingleLineComment
: '//' ~( '\r' | '\n' )* {setText("replacement");}
;
What i want to do is replace all single-line comments in a file, let's say.
Rewrite the text inside the lexer:
SingleLineComment
: '//' ~('\r' | '\n')* {setText("replacement");}
;
EDIT
Okay, here's a quick demo how you can filter certain tokens from a language:
SingleCommentStrip.g
grammar SingleCommentStrip;
parse returns [String str]
@init{StringBuilder builder = new StringBuilder();}
: (t=. {builder.append($t.text);})* EOF {$str = builder.toString();}
;
SingleLineComment
: '//' ~('\r' | '\n')* {skip();}
;
MultiLineComment
: '/*' .* '*/'
;
StringLiteral
: '"' ('\\' . | ~('"' | '\\' | '\r' | '\n'))* '"'
;
AnyOtherChar
: .
;
Main.java
import org.antlr.runtime.*;
public class Main {
public static void main(String[] args) throws Exception {
SingleCommentStripLexer lexer = new SingleCommentStripLexer(new ANTLRFileStream("Test.java"));
SingleCommentStripParser parser = new SingleCommentStripParser(new CommonTokenStream(lexer));
String adjusted = parser.parse();
System.out.println(adjusted);
}
}
Test.java
// COMMENT
class Test {
/*
// don't remove
*/
// COMMENT AS WELL
String s = "/* don't // remove */ \" \\ me */ as well";
}
Now run the demo:
java -cp antlr-3.3.jar org.antlr.Tool SingleCommentStrip.g
javac -cp antlr-3.3.jar *.java
java -cp .:antlr-3.3.jar Main
which will print:
class Test { /* // don't remove */ String s = "/* don't // remove */ \" \\ me */ as well"; }
(i.e. the single line comments are removed)
这篇关于替换 ANTLR 中的令牌的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!