ANTLR3 C 目标 - 解析器返回“错过"根元素 [英] ANTLR3 C Target - parser return 'misses' out root element

问题描述

我正在尝试使用 ANTLR3 C 目标来理解 AST，但遇到了一些困难.

I'm trying to use the ANTLR3 C Target to make sense of an AST, but am running into some difficulties.

我有一个简单的类似 SQL 的语法文件:

I have a simple SQL-like grammar file:

grammar sql;
options 
{
    language = C;
    output=AST;
    ASTLabelType=pANTLR3_BASE_TREE; 
}
sql :   VERB fields;
fields  :   FIELD (',' FIELD)*;
VERB    :   'SELECT' | 'UPDATE' | 'INSERT';
FIELD   :   CHAR+;
fragment
CHAR    :   'a'..'z';

这在 ANTLRWorks 中按预期工作.

and this works as expected within ANTLRWorks.

在我的 C 代码中，我有:

In my C code I have:

const char pInput[] = "SELECT one,two,three";
pANTLR3_INPUT_STREAM pNewStrm = antlr3NewAsciiStringInPlaceStream((pANTLR3_UINT8) pInput,sizeof(pInput),NULL);
psqlLexer lex =  sqlLexerNew         (pNewStrm);
pANTLR3_COMMON_TOKEN_STREAM   tstream = antlr3CommonTokenStreamSourceNew(ANTLR3_SIZE_HINT,
    TOKENSOURCE(lex));
psqlParser ps = sqlParserNew( tstream );
sqlParser_sql_return ret = ps->sql(ps);
pANTLR3_BASE_TREE pTree = ret.tree;
cout << "Tree: " << pTree->toStringTree(pTree)->chars << endl;
ParseSubTree(0,pTree);

当您使用 ->getChildCount 和 ->children->get 递归遍历树时，这会输出一个扁平的树结构.

This outputs a flat tree structure when you use ->getChildCount and ->children->get to recurse through the tree.

void ParseSubTree(int level,pANTLR3_BASE_TREE pTree)
{
    ANTLR3_UINT32 childcount =  pTree->getChildCount(pTree);

    for (int i=0;i<childcount;i++)
    {
        pANTLR3_BASE_TREE pChild = (pANTLR3_BASE_TREE) pTree->children->get(pTree->children,i);
        for (int j=0;j<level;j++)
        {
            std::cout << " - ";
        }
        std::cout << 
            pChild->getText(pChild)->chars <<       
            std::endl;
        int f=pChild->getChildCount(pChild);
        if (f>0)
        {
            ParseSubTree(level+1,pChild);
        }
    }
}

程序输出:树:选择一、二、三选择一,二,三

Program output: Tree: SELECT one , two , three SELECT one , two , three

现在，如果我改变语法文件:

Now, if I alter the grammar file:

sql :   VERB ^fields;

.. 对 ParseSubTree 的调用只显示字段的子节点.

.. the call to ParseSubTree only displays the child nodes of fields.

程序输出:树:(选择一、二、三)一,二,三

Program output: Tree: (SELECT one , two , three) one , two , three

我的问题是:为什么在第二种情况下，Antlr 只是给子节点?(实际上错过了 SELECT 标记)如果有人能给我任何指示以了解 Antlr 返回的树，我将不胜感激.

My question is: why, in the second case, is Antlr just give the child nodes? (in effect missing out the SELECT token) I'd be very grateful if anybody can give me any pointers for making sense of the tree returned by Antlr.

有用信息:AntlrWorks 1.4.2，Antlr C 目标 3.3，MSVC 10

Useful Information: AntlrWorks 1.4.2, Antlr C Target 3.3, MSVC 10

推荐答案

在选项部分放置 output=AST; 不会产生实际的 AST，它只会导致 ANTLR 创建 CommonTree 标记而不是 CommonTokens (或者，在您的情况下，等效的 C 结构).

Placing output=AST; in the options section will not produce an actual AST, it only causes ANTLR to create CommonTree tokens instead of CommonTokens (or, in your case, the equivalent C structs).

如果您使用 output=AST;，下一步是将树操作符放入，或在解析器规则中重写规则，以形成 AST.

If you use output=AST;, the next step is to put tree operators, or rewrite rules inside your parser rules that give shape to your AST.

请参阅此之前的问答以了解详情如何创建合适的 AST.

See this previous Q&A to find out how to create a proper AST.

例如下面的语法(有重写规则):

For example, the following grammar (with rewrite rules):

options {
  output=AST;
  // ...
}

sql                        // make VERB the root
  :  VERB fields        -> ^(VERB fields) 
  ;

fields                     // omit the comma's from the AST
  :  FIELD (',' FIELD)* -> FIELD+
  ;

VERB  : 'SELECT' | 'UPDATE' | 'INSERT';
FIELD : CHAR+;
SPACE : ' ' {$channel=HIDDEN;};
fragment CHAR : 'a'..'z';

将解析以下输入:

UPDATE         field,     foo  ,  bar

进入以下AST:

这篇关于ANTLR3 C 目标 - 解析器返回“错过"根元素的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！