Haskell词法布局规则的实现 [英] Haskell lexical layout rule implementation

问题描述

我一直在研究一种带有Haskell语法的宠物语言。 Haskell做的其中一件我一直试图复制的东西是插入{，}和;在解析步骤之前，基于代码布局的令牌。

我发现 http://www.haskell.org/onlinereport/syntax-iso.html ，其中包括如何实现布局程序的规范，并制作了一个版本（当然，修改为我的（更简单的）语言）。

不幸的是，我得到了一个不正确的输出：

f（do xyz）ab

它应该产生标记流 ID（DO {ID ID ID}）ID ID ，但它会产生令牌流 ID（DO {ID ID ID）ID ID} 。

我想这是因为我对 parse-error（t）（ parse-error（t）= false ），但我不知道如何有效地执行 parse-error（t）。

GHC等Haskell编译器如何处理这种情况？是否有一种简单的方法来实现 parse-error（t），这样它就可以处理这种情况（并且希望其他人我还没有注意到）？

$我最终实现了由JISON编译的解析器使用的解析算法的定制版本，该解析器采用不可变状态对象和令牌，并执行在返回之前尽可能多地使用令牌工作。然后我可以使用这个实现来检查一个标记是否会产生一个分析错误，并且很容易回滚到解析器的以前的状态。

它工作的很好，尽管现在它有点黑客。你可以在这里找到代码： https://github.com/mystor/ myst / blob / cb9b7b7d83e5d00f45bef0f994e3a4ce71c11bee / compiler / layout.js

我试着做@augustss建议的东西，使用错误产生来伪造令牌的插入，但它看起来好像JISON没有我需要的所有工具来获得可靠的实现，并且重新实现分解算法的简化版本变得更容易，并且与原始文档更好地排列在一起。

I have been working on a pet language, which has Haskell-like syntax. One of the neat things which Haskell does, which I have been trying to replicate, is its insertion of {, } and ; tokens based on code layout, before the parsing step.

I found http://www.haskell.org/onlinereport/syntax-iso.html, which includes a specification of how to implement the layout program, and have made a version of it (modified, of course, for my (much simpler) language).

Unfortunately, I am getting an incorrect output for the following:

f (do x y z) a b

It should be producing the token stream ID ( DO { ID ID ID } ) ID ID, but it is instead producing the token stream ID ( DO { ID ID ID ) ID ID }.

I imagine that this is due to my unsatisfactory implementation of parse-error(t) (parse-error(t) = false), but I have no idea how I would go about implementing parse-error(t) efficiently.

How do Haskell compilers like GHC etc. handle this case? Is there an easy way to implement parse-error(t) such that it handles this case (and hopefully others that I haven't noticed yet)?

解决方案

I ended up implementing a custom version of the parsing algorithm used by JISON's compiled parsers, which takes an immutable state object, and a token, and performs as much work as possible with the token before returning. I am then able to use this implementation to check if a token will produce a parse error, and easily roll back to previous states of the parser.

It works fairly well, although it is a bit of a hack right now. You can find the code here: https://github.com/mystor/myst/blob/cb9b7b7d83e5d00f45bef0f994e3a4ce71c11bee/compiler/layout.js

I tried doing what @augustss suggested, using the error production to fake the token's insertion, but it appears as though JISON doesn't have all of the tools which I need in order to get a reliable implementation, and re-implementing a stripped-down version of the parsing algorithm turned out to be easier, and lined up better with the original document.

这篇关于Haskell词法布局规则的实现的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！