ANTLR PCRE语法到JS目标 [英] ANTLR PCRE Grammar to JS Target
问题描述
我正在尝试构建Bart Kiers的ANTLR PCRE语法(请参阅: http ://big-o.nl/apps/pcreparser/pcre/PCREParser.html )添加到JS目标.我得到它构建的唯一方法是使用全局回溯和记忆，并且生成的代码在这里是无效的，即语法:
I'm trying to build Bart Kiers' ANTLR PCRE grammar (see: http://big-o.nl/apps/pcreparser/pcre/PCREParser.html) to a JS target. The only way I get get it to build is with global backtracking and memoization and it's generating code that is invalid here is the grammar:
grammar PCRE;
options {
language=JavaScript;
backtrack=true;
memoize=true;
}
parse
: regexAtom* EOF
;
... and the rest of the grammar as seen: http://big-o.nl/apps/pcreparser/pcre/PCREParser.html
词法分析器正在生成类似以下代码的代码:
The lexer is generating code that looks like:
//$ANTLR 3.4 PCRE.g 2011-11-19 23:22:35
var PCRELexer = function(input, state) {
// alternate constructor @todo
// public PCRELexer(CharStream input)
// public PCRELexer(CharStream input, RecognizerSharedState state) {
if (!state) {
state = new org.antlr.runtime.RecognizerSharedState();
}
(function(){
}).call(this);
PCRELexer.superclass.constructor.call(this, input, state);
};
org.antlr.lang.augmentObject(PCRELexer, {
: ,
: ,
: ,
: ,
: ,
... and more of this empty object.
当我尝试使用此生成的代码时，我在上面的stevementObject行上遇到了JS错误.有人可以提供有关如何使它针对JS目标正确构建的说明.我最终将构建一个walker，以生成与同一页面上显示的输出类似的输出.
And when I try to use this generated code I'm getting JS errors on the arguementObject line above. Can someone provide direction of how I can get this to build correctly for a JS target. I'm will ultimately build a walker to generate similar output to the output shown on the same page.
推荐答案
更新:可在此处找到生成JS lexer-和解析器文件的语法: https://github.com/bkiers/PCREParser/tree/js
我已经有一段时间没有更新该页面了(如果有时间的话，我会尽快更新).这是修改后的语法，它不使用全局回溯，并且可以(据我所知)与JavaScript一起使用(经ANTLR v3.3测试):
I haven't updated that page in a while (I will do so soon, if I find the time). Here's the modified grammar that doesn't use global backtracking, and works (as far as I can see) with JavaScript (tested with ANTLR v3.3):
grammar PCRE;
options {
output=AST;
language=JavaScript;
}
tokens {
REGEX;
ATOM;
DOT;
OR;
CHAR_CLASS;
NEG_CHAR_CLASS;
RANGE;
QUOTATION;
INT;
QUANTIFIER;
GREEDY;
RELUCTANT;
POSSESSIVE;
BACK_REFERENCE;
CAPTURE_GROUP;
FLAG_GROUP;
ATOMIC_GROUP;
NON_CAPTURE_GROUP;
POSITIVE_LOOK_AHEAD;
NEGATIVE_LOOK_AHEAD;
POSITIVE_LOOK_BEHIND;
NEGATIVE_LOOK_BEHIND;
FLAGS;
ENABLE;
DISABLE;
DOT;
ATOM;
ATOMS;
}
// parser rules
parse
: regexAtoms EOF -> ^(REGEX regexAtoms)
;
regexAtoms
: atoms (Or^ atoms)*
;
atoms
: regexAtom* -> ^(ATOMS regexAtom*)
;
regexAtom
: unit quantifier? -> ^(ATOM unit quantifier?)
;
unit
: charClass
| singleChar
| boundaryMatch
| quotation
| backReference
| group
| ShorthandCharacterClass
| PosixCharacterClass
| Dot
;
quantifier
: (greedy -> ^(GREEDY greedy))
('+' -> ^(POSSESSIVE greedy)
|'?' -> ^(RELUCTANT greedy)
)?
;
greedy
: '+' -> INT["1"] INT["2147483647"]
| '*' -> INT["0"] INT["2147483647"]
| '?' -> INT["0"] INT["1"]
| '{' (a=integer -> INT[$a.text] INT[$a.text])
(
(',' -> INT[$a.text] INT["2147483647"])
(b=integer -> INT[$a.text] INT[$b.text])?
)?
'}'
;
charClass
: '[' (('^')=> '^' charClassAtom+ ']' -> ^(NEG_CHAR_CLASS charClassAtom+)
| charClassAtom+ ']' -> ^(CHAR_CLASS charClassAtom+)
)
;
charClassAtom
: (charClassSingleChar '-' charClassSingleChar)=>
charClassSingleChar '-' charClassSingleChar -> ^(RANGE charClassSingleChar charClassSingleChar)
| quotation
| ShorthandCharacterClass
| BoundaryMatch
| PosixCharacterClass
| charClassSingleChar
;
charClassSingleChar
: charClassEscape
| EscapeSequence
| OctalNumber
| SmallHexNumber
| UnicodeChar
| Or
| Caret
| Hyphen
| Colon
| Dollar
| SquareBracketStart
| RoundBracketStart
| RoundBracketEnd
| CurlyBracketStart
| CurlyBracketEnd
| Equals
| LessThan
| GreaterThan
| ExclamationMark
| Comma
| Plus
| Star
| QuestionMark
| Dot
| Digit
| OtherChar
;
charClassEscape
: '\\' ('\\' | '^' | ']' | '-')
;
singleChar
: regexEscape
| EscapeSequence
| OctalNumber
| SmallHexNumber
| UnicodeChar
| Hyphen
| Colon
| SquareBracketEnd
| CurlyBracketEnd
| Equals
| LessThan
| GreaterThan
| ExclamationMark
| Comma
| Digit
| OtherChar
;
regexEscape
: '\\' ('\\' | '|' | '^' | '$' | '[' | '(' | ')' | '{' | '}' | '+' | '*' | '?' | '.')
;
boundaryMatch
: Caret
| Dollar
| BoundaryMatch
;
backReference
: '\\' integer -> ^(BACK_REFERENCE integer)
;
group
: '('
( '?' ( (flags -> ^(FLAG_GROUP flags)
)
(':' regexAtoms -> ^(NON_CAPTURE_GROUP flags regexAtoms)
)?
| '>' regexAtoms -> ^(ATOMIC_GROUP regexAtoms)
| '!' regexAtoms -> ^(NEGATIVE_LOOK_AHEAD regexAtoms)
| '=' regexAtoms -> ^(POSITIVE_LOOK_AHEAD regexAtoms)
| '<' ( '!' regexAtoms -> ^(NEGATIVE_LOOK_BEHIND regexAtoms)
| '=' regexAtoms -> ^(POSITIVE_LOOK_BEHIND regexAtoms)
)
)
| regexAtoms -> ^(CAPTURE_GROUP regexAtoms)
)
')'
;
flags
: (a=singleFlags -> ^(FLAGS ^(ENABLE $a)))
('-' b=singleFlags -> ^(FLAGS ^(ENABLE $a) ^(DISABLE $b))
)?
;
singleFlags
: OtherChar*
;
quotation
: QuotationStart innerQuotation QuotationEnd -> ^(QUOTATION innerQuotation)
;
innerQuotation
: (~QuotationEnd)*
;
integer
: (options{greedy=true;}: Digit)+
;
// lexer rules
QuotationStart
: '\\Q'
;
QuotationEnd
: '\\E'
;
PosixCharacterClass
: '\\p{' ('Lower' | 'Upper' | 'ASCII' | 'Alpha' | 'Digit' | 'Alnum' | 'Punct' | 'Graph' | 'Print' | 'Blank' | 'Cntrl' | 'XDigit' | 'Space') '}'
;
ShorthandCharacterClass
: Escape ('d' | 'D' | 's' | 'S' | 'w' | 'W')
;
BoundaryMatch
: Escape ('b' | 'B' | 'A' | 'Z' | 'z' | 'G')
;
OctalNumber
: Escape '0' ( OctDigit? OctDigit
| '0'..'3' OctDigit OctDigit
)
;
SmallHexNumber
: Escape 'x' HexDigit HexDigit
;
U nicodeChar
: Escape 'u' HexDigit HexDigit HexDigit HexDigit
;
EscapeSequence
: Escape ('t' | 'n' | 'r' | 'f' | 'a' | 'e' | ~('a'..'z' | 'A'..'Z' | '0'..'9'))
;
Escape : '\\';
Or : '|';
Hyphen : '-';
Caret : '^';
Colon : ':';
Dollar : '$';
SquareBracketStart : '[';
SquareBracketEnd : ']';
RoundBracketStart : '(';
RoundBracketEnd : ')';
CurlyBracketStart : '{';
CurlyBracketEnd : '}';
Equals : '=';
LessThan : '<';
GreaterThan : '>';
ExclamationMark : '!';
Comma : ',';
Plus : '+';
Star : '*';
QuestionMark : '?';
Dot : '.';
Digit : '0'..'9';
OtherChar : . ;
// fragments
fragment OctDigit : '0'..'7';
fragment HexDigit : ('0'..'9' | 'a'..'f' | 'A'..'F');
它几乎没有目标特定的代码.我唯一要做的就是使用几个字符串文字来重写AST(请参阅量词)和几个.text调用，但是几乎所有ANTLR目标都接受双引号字符串文字和.text，因此您应该对Java没问题，Python，C和JavaScript.对于C#，我想您需要将.text调用更改为.Text.
It contains next to no target specific code. The only thing I did is use a couple of string literals to rewrite AST's (see the quantifier) and a few .text calls, but almost all ANTLR targets accept double quoted string literals and .text, so you should be okay with Java, Python, C and JavaScript. For C#, I guess you'd need to change the .text invocations to .Text.
您可以使用以下HTML文件对其进行测试:
You can test it with the following HTML file:
<html>
<head>
<script type="text/javascript" src="antlr3-all-min.js"></script>
<script type="text/javascript" src="PCRELexer.js"></script>
<script type="text/javascript" src="PCREParser.js"></script>
<style type="text/css">
#tree {
padding: 20px;
font-family: Monospace;
}
.leaf {
font-weight: bold;
font-size: 130%;
}
</style>
<script type="text/javascript">
function init() {
document.getElementById("parse").onclick = parseRegex;
}
function parseRegex() {
document.getElementById("tree").innerHTML = "";
var regex = document.getElementById("regex").value;
if(regex) {
var lexer = new PCRELexer(new org.antlr.runtime.ANTLRStringStream(regex));
var parser = new PCREParser(new org.antlr.runtime.CommonTokenStream(lexer));
var root = parser.parse().getTree();
printTree(root, 0);
}
else {
document.getElementById("regex").value = "enter a regex here first";
}
}
function printTree(root, indent) {
if(!root) return;
for(var i = 0; i < indent; i++) {
document.getElementById("tree").innerHTML += ". ";
}
var n = root.getChildCount();
if(n == 0) {
document.getElementById("tree").innerHTML += "<span class=\"leaf\">" + root + "</span><br />";
}
else {
document.getElementById("tree").innerHTML += root + "<br />";
}
for(i = 0; i < n; i++) {
printTree(root.getChild(i), indent + 1);
}
}
</script>
</head>
<body onload="init()">
<input id="regex" type="text" size="50" />
<button id="parse">parse</button>
<div id="tree"></div>
</body>
</html>
(我很少使用JavaScript，所以不要介意上面的烂摊子！)
如果您现在解析正则表达式:
If you now parse the regex:
[^-234-7]|(?=[ab\]@]++$).|^$|\1\.\(
借助上面的HTML文件，您将看到以下内容打印到屏幕上:
with the help of the HTML file above, you will see the following being printed to the screen:
REGEX
. |
. . |
. . . |
. . . . ATOMS
. . . . . ATOM
. . . . . . NEG_CHAR_CLASS
. . . . . . . -
. . . . . . . 2
. . . . . . . 3
. . . . . . . RANGE
. . . . . . . . 4
. . . . . . . . 7
. . . . ATOMS
. . . . . ATOM
. . . . . . POSITIVE_LOOK_AHEAD
. . . . . . . ATOMS
. . . . . . . . ATOM
. . . . . . . . . CHAR_CLASS
. . . . . . . . . . a
. . . . . . . . . . b
. . . . . . . . . . \]
. . . . . . . . . . @
. . . . . . . . . POSSESSIVE
. . . . . . . . . . 1
. . . . . . . . . . 2147483647
. . . . . . . . ATOM
. . . . . . . . . $
. . . . . ATOM
. . . . . . .
. . . ATOMS
. . . . ATOM
. . . . . ^
. . . . ATOM
. . . . . $
. . ATOMS
. . . ATOM
. . . . BACK_REFERENCE
. . . . . 1
. . . ATOM
. . . . \.
. . . ATOM
. . . . \(请注意，我没有正确测试语法！如果您发现有任何错误，请告诉我，我们将不胜感激.
Be careful, I have not tested the grammar properly! I'd appreciate it if you let me know if you find any bugs in it.
如果取消注释language=JavaScript;行，请重新生成词法分析器和解析器并运行以下类:
If you un-comment the line language=JavaScript;, regenerate a lexer and parser and run the following class:
import org.antlr.runtime.*;
import org.antlr.runtime.tree.*;
import org.antlr.stringtemplate.*;
public class Main {
public static void main(String[] args) throws Exception {
String src = "[^-234-7]|(?=[ab\\]@]++$).|^$|\\1\\.\\(|\\Q*+[\\E";
PCRELexer lexer = new PCRELexer(new ANTLRStringStream(src));
PCREParser parser = new PCREParser(new CommonTokenStream(lexer));
CommonTree tree = (CommonTree)parser.parse().getTree();
DOTTreeGenerator gen = new DOTTreeGenerator();
StringTemplate st = gen.toDOT(tree);
System.out.println(st);
}
}
您将看到与以下AST对应的DOT输出:
you'll see DOT-output that corresponds to the following AST:
这篇关于ANTLR PCRE语法到JS目标的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
