ANTLR v3:订单令牌以提高 tree walker 的性能 [英] ANTLR v3: order token to improve performance of tree walker

问题描述

是否可以以某种方式指定 ANTLR v3 生成的令牌的顺序?我的目标是以这样一种方式对标记进行排序，即我的规则表达式"中的有效标记相互跟随，以便树遍历器(具有大约 90 个不同的分支)中的条件可以简化为一个分支.类似的东西

Is it somehow possible to specify the order of the tokens generated by ANTLR v3? My goal is to order the tokens in such a way that valid tokens in my rule "expression" follow each other in order that conditions in the tree walkers (which have about 90 different branches) can be simplified to one branch. Something like

if(LA18_0 >= ARRAY_ACCESS && LA18_0 <= VariableId){}

ANTLR 按字母顺序将值分配给标记.这意味着，以a"开头的标记具有最低值，以z"开头的标记最高.

ANTLR assigns the values to the tokens in alphabetical order. That means, the token beginning with an "a" has the lowest value, token with "z" the highest.

一个例子来说明我的问题.抽象"(在我的规则表达式"中不是有效标记)在 ARRAY_ACCESS(其值为 4)之后的值为 5，因此条件必须如下所示:

An example to illustrate my problem. "abstract" (which is not a valid token in my rule "expression") has value 5 right after ARRAY_ACCESS (which has value 4) and thus the condition has to look like the following:

if(LA18_0 == ARRAY_ACCESS || (LA18_0 >= Assign && LA18_0 <= VariableId)){}

这只是一个例子，正如我所说，大约有 90 个分支机构.

That's just an example, as I said, there are about 90 branches.

我可以在这些标记前加上前缀来实现我的目标，但这看起来相当难看并且妨碍了可读性.还有别的方法吗?

I could prefix those tokens to achieve my goal but that looks rather ugly and hampers the readability. Is there another way?

推荐答案

保证 ANTLR 对特定标记使用特定常量的唯一方法是编写您自己的 .tokens 文件(您将看到当您编译语法时，它们会生成 ANTLR).然后，您可以使用 tokenVocab 选项指定自定义令牌文件的名称.

The only way to guarantee that ANTLR uses particular constants for particular tokens is to write your own .tokens file (you'll see them produced ANTLR when you compile grammars). You can then use the tokenVocab option to specify the name of your custom tokens file.

也就是说，根据我的经验，令牌的排序从未被证明是一个重要的性能问题.ANTLR 3 的 C# 端口通过消除出现在某些代码路径上的冗余范围检查来解决性能问题.

That said, the ordering of tokens has never proven to be a significant performance issue in my experience. The C# port of ANTLR 3 approaches the performance issue by eliminating redundant range checks that appear on some code paths.

这篇关于ANTLR v3:订单令牌以提高 tree walker 的性能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！