语义词法分析器谓词性能 [英] Semantic lexer predicate performance

问题描述

我有一个词法分析器为传递给词法分析器的宏字符串的动态列表创建宏标记.我在最顶级的词法分析器规则中使用了语义谓词来实现此功能:

I have a lexer creates MACRO tokens for a dynamic list of macro strings passed to the lexer. I used a semantic predicate in the very top lexer rule to implement this feature:

MACRO: { macros != null && tryMacro() }? .;

其中 tryMacro() 只是检查是否有任何宏字符串与输入序列匹配.

Where tryMacro() just checks if any macro string matches the input sequence.

这种方法的性能非常糟糕，经过一些研究，我尝试将词法分析器规则更改为以下内容:

The performance of this approach was very bad and after some research I tried changing the lexer rule to the following:

MACRO: . { macros != null && tryMacro() }?;

这大大提高了性能，但我真的不明白为什么.:) 由于 '.'匹配任何字符，语义谓词规则应该像以前一样被调用多次，不是吗?有人可以解释这种行为吗?

This severely improved the performance but I don't really understand why. :) Since the '.' matches any character, the semantic predicate rule should be invoked exactly as many times as before, shouldn't it? Can someone provide an explanation for this behavior?

推荐答案

我进一步调试了这一点，结果发现规则的重新排序改变了词法分析器的行为，导致在解析过程中不接受宏.感知性能提高的原因是因为语义谓词在词法分析器在进行预测时放弃规则之前只评估了几次.所以规则的改变实际上是无效的，而不是性能的提升.

I debugged this a bit further and it turned out that the reordering of the rule changed the behavior of the lexer causing macros to not be accepted during parsing. The reason for the perceived increase in performance was because the semantic predicate was only evaluated a couple of times before the lexer dropped the rule while doing its predictions. So the change of the rule was actually invalid and not a performance improvement.

我终于通过将宏处理移到解析器来解决性能问题.

I finally solved the performance issue by moving the macro handling to the parser.

这篇关于语义词法分析器谓词性能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！