使用ANTLR4设计灵活语言应用程序的一般策略 [英] General stategy for designing Flexible Language application using ANTLR4

问题描述

我正在尝试使用antlr4开发语言应用程序.所讨论的语言并不重要.重要的是语法非常广泛(轻松地> 2000条规则！).我想做一些操作

I am trying to develop a language application using antlr4. The language in question is not important. The important thing is that the grammar is very vast (easily >2000 rules!!!). I want to do a number of operations

• 提取大量信息.这些可以是调用图，变量名.常数表达式等.
• 任意数量的转换:
  • 如果可以扩展循环，我们继续进行扩展
  • 如果我们可以消除无效代码，我们可以选择这样做
  • 我们可能会选择重命名所有变量名称以符合某些规范.
  • Extract bunch of informations. These can be call graphs, variable names. constant expressions etc.
  • Any number of transformations:
    • if a loop can be expanded, we go ahead and expand it
    • If we can eliminate dead code we might choose to do that
    • we might choose to rename all variable names to conform to some norms.

    这些操作中的每一个都可以彼此独立地应用.在应用了这些步骤之后，我希望重写输入尽可能接近原始输入.

    Each of these operations can be applied independent of each other. And after application of these steps I want the rewrite the input as close as possible to the original input.

    例如因此，我们可能希望消除循环并重命名变量，然后以原始语言格式输出结果.

    e.g. So we might want to eliminate loops and rename the variable and then output the result in the original language format.

    1. 我认为有必要为此构建自定义树(阅读AST).这样我就可以通过每次转换来修改树.但是，当我想生成输出时，我失去了 TokenStreamRewriter 的出色功能.我必须指定如何编写树的每个节点，并且我对未进行任何转换的位置失去了原始输入格式. antlr4是否提供解决此问题的好方法?
    2. AST是最好的选择吗?还是建立自己的对象表示形式?如果是这样，我如何有效地创建该对象?对于如此广泛的语言而言，创建对象表示形式是非常痛苦的.但从长远来看可能会更好.再次如何恢复原始格式?
    3. 是否可以仅在解析树上工作?
    4. 是否存在类似的语言应用程序在做相同的事情?如果是的话，他们会使用什么策略?
    1. I see a need to build a custom Tree (read AST) for this. So that I can modify the tree with each of the transformations. However when I want to generate the output, I lose the nice abilities of the TokenStreamRewriter. I have to specify how to write each of the nodes of the tree and I lose the original input formatting for the places I didn't do any transformations. Does antlr4 provide a good way to get around this problem?
    2. Is AST the best way to go? Or do I build my own object representation? If so how do I create that object efficiently? Creating object representation is very big pain for such a vast language. But may be better in the long run. Again how do I get back the original formatting?
    3. Is it possible to work just on the parse tree?
    4. Are there similar language applications which do the same thing? If so what strategy do they use?

    欢迎任何输入. 提前致谢.

    Any input is welcome. Thanks in advance.

    推荐答案

    通常，您想要的称为程序转换系统(PTS).

    PTS通常具有解析器，构建AST，可以对AST进行漂亮打印以恢复可编译的源文本.更重要的是，它们具有导航/检查/修改AST的标准方法，因此您可以以编程方式对其进行更改.

    PTSs generally have parsers, build ASTs, can prettyprint the ASTs to recover compilable source text. More importantly, they have standard ways to navigate/inspect/modify the ASTs so that you can change them programmatically.

    许多功能都以模式匹配的代码片段的形式提供，这些代码片段以要转换的语言的表面语法编写；这避免了永远需要了解有关AST中的哪些节点以及它们与子级之间的关系的极其精细的细节.当您处理复杂的语法时，这是非常有用的，因为我们大多数现代语言(以及我们的传统语言)似乎都有.

    Many offer these capabilities in the form of pattern-matching code fragments written in the surface syntax of the language being transformed; this avoids the need to forever having to know excruciatingly fine details about which nodes are in your AST and how they are related to children. This is incredibly useful when you big complex grammars, as most of our modern (and our legacy languages) all seem to have.

    更复杂的PTS(很少)提供了额外的工具，可用来调解源代码的语义.在不知道各个符号属于什么范围，它们的类型以及许多其他细节(例如数据流)的情况下，很难分析/转换大多数代码.完全公开:我将其中之一.

    More sophisticated PTSs (very few) provide additional facilities for teasing out the semantics of the source code. It is pretty hard to analyze/transform most code without knowing what scopes individual symbols belong to, or their type, and many other details such as data flow. Full disclosure: I build one of these.

    这篇关于使用ANTLR4设计灵活语言应用程序的一般策略的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！