在ANTLR的3C令牌的返回值 [英] Token return values in ANTLR 3 C
问题描述
我是新来ANTLR,而我试图写用C语言的目标(antler3C)一个简单的解析器。语法很简单,我想有每个规则返回一个值,例如:
I'm new to ANTLR, and I'm attempting to write a simple parser using C language target (antler3C). The grammar is simple enough that I'd like to have each rule return a value, eg:
number returns [long value]
:
( INT {$value = $INT.ivalue;}
| HEX {$value = $HEX.hvalue;}
)
;
HEX returns [long hvalue]
: '0' 'x' ('0'..'9'|'a'..'f'|'A'..'F')+ {$hvalue = strtol((char*)$text->chars,NULL,16);}
;
INT returns [long ivalue]
: '0'..'9'+ {$ivalue = strtol((char*)$text->chars,NULL,10);}
;
每个规则收集它的返回值是孩子的规则,直到最顶端的规则返回一个很好的结构充满我的数据。
Each rule collects the return value of it's child rules until the topmost rule returns a nice struct full of my data.
据我所知,ANTLR允许词法规则(令牌,例如'INT'和;HEX)返回值一样的语法规则(例如'号')。然而,生成的C code将不能编译:
As far as I can tell, ANTLR allows lexer rules (tokens, eg 'INT' & 'HEX') to return values just like parser rules (eg 'number'). However, the generated C code will not compile:
error C2228: left of '.ivalue' must have class/struct/union
error C2228: left of '.hvalue' must have class/struct/union
我做了一些闲逛,和错误意义 - 令牌最终成为通用ANTLR3_COMMON_TOKEN_struct,它不允许返回值。所以,也许Ç目标只是不支持此功能。但是就像我说的,我是新来这个,我去哈林去寻找另一种方法,我想确认,我不能做这种方式了。
I did some poking around, and the errors make sense - the tokens end up as generic ANTLR3_COMMON_TOKEN_struct, which doesn't allow for a return value. So maybe the C target just doesn't support this feature. But like I said, I'm new to this, and before I go haring off to find another approach I want to confirm that I can't do it this way.
所以,问题是这样的:是否antler3C支持返回值词法规则,若然是使用它们的正确方法?
So the question is this: 'Does antler3C support return values for lexer rules, and if so what is the proper way to use them?'
推荐答案
不是真的任何新信息,只是在什么@bemace已经提到了一些细节。
Not really any new information, just some details on what @bemace already mentioned.
没有,词法规则不能有返回值。请参阅 4.3规则 权威ANTLR参考:
No, lexer rules cannot have return values. See 4.3 Rules from The Definitive ANTLR reference:
就像函数调用,ANTLR解析器和树解析器规则可以有
参数和返回值。 ANTLR词法规则不能有回报
值[...]
Rule Arguments and Return Values
Just like function calls, ANTLR parser and tree parser rules can have arguments and return values. ANTLR lexer rules cannot have return values [...]
有两种选择:
您可以做转换到长
在解析器规则数量
:
You can do the transforming to a long
in the parser rule number
:
number returns [long value]
: INT {$value = Long.parseLong($INT.text);}
| HEX {$value = Long.parseLong($HEX.text.substring(2), 16);}
;
选项2
或者创建自己的令牌有,比如说,一个 toLong():长
方法:
import org.antlr.runtime.*;
public class YourToken extends CommonToken {
public YourToken(CharStream input, int type, int channel, int start, int stop) {
super(input, type, channel, start, stop);
}
// your custom method
public long toLong() {
String text = super.getText();
int radix = text.startsWith("0x") ? 16 : 10;
if(radix == 16) text = text.substring(2);
return Long.parseLong(text, radix);
}
}
,并在选项{...}
头在你的语法使用此令牌并覆盖发出()定义:令牌
法在词法分析器类:
and define in the options {...}
header in your grammar to use this token and override the emit(): Token
method in your lexer class:
grammar Foo;
options{
TokenLabelType=YourToken;
}
@lexer::members {
public Token emit() {
YourToken t = new YourToken(input, state.type, state.channel,
state.tokenStartCharIndex, getCharIndex()-1);
t.setLine(state.tokenStartLine);
t.setText(state.text);
t.setCharPositionInLine(state.tokenStartCharPositionInLine);
emit(t);
return t;
}
}
parse
: number {System.out.println("parsed: "+$number.value);} EOF
;
number returns [long value]
: INT {$value = $INT.toLong();}
| HEX {$value = $HEX.toLong();}
;
HEX
: '0' 'x' ('0'..'9'|'a'..'f'|'A'..'F')+
;
INT
: '0'..'9'+
;
当你生成一个解析器和词法分析器,并运行这个测试类:
When you generate a parser and lexer, and run this test class:
import org.antlr.runtime.*;
import java.io.*;
public class Main {
public static void main(String[] args) throws Exception {
ANTLRStringStream in = new ANTLRStringStream("0xCafE");
FooLexer lexer = new FooLexer(in);
CommonTokenStream tokens = new CommonTokenStream(lexer);
FooParser parser = new FooParser(tokens);
parser.parse();
}
}
这将产生以下的输出:
it will produce the following output:
parsed: 51966
第一个选择似乎更实用于你的情况。
The first options seems the more practical in your case.
需要注意的是,你可以看到,给出的例子是Java。我不知道如果选择2在C目标/运行时支持。我决定还是发布它可以在这里用它作为日后参考上左右。
Note that, as you can see, the examples given are in Java. I have no idea if option 2 is supported in the C target/runtime. I decided to still post it to be able to use it as a future reference here on SO.
这篇关于在ANTLR的3C令牌的返回值的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!