Haskell有反向模式自动区分的实现吗？ [英] Is there any working implementation of reverse mode automatic differentiation for Haskell?

问题描述

在我看到的Haskell中最接近的实现是在 http://hackage.haskell.org/packages/archive/fad/1.0/doc/html/Numeric-FAD.html 。

最接近的相关研究似乎是另一种与Scheme相关的功能语言的反向模式， http://www.bcl.hamilton.ie/~qobi/stalingrad/ 。

我在Haskell中看到反向模式是一种圣杯对于很多任务而言，希望它能够使用Haskell的嵌套数据并行机制，以便在重度数值优化中获得不错的加速比。 解决方案

p>为了回应这个问题，我上传了一个名为广告的软件包给Hackage以处理反向Ha模式自动分化

在内部，它利用Andy Gill的堪萨斯熔岩中的一个技巧来观察它记录的磁带中的共享以用于反向传播目的，并使用类型级别的品牌来避免混淆敏感度。

我试着让API与Barak Pearlmutter和Jeffrey Mark Siskind的时尚包保持相对接近，但我忍不住做了一些小小的调整

我仍然需要完成剩余的未实现的时尚组合器，找出构建反向模式AD塔的好方法，验证我没有搞砸我对基本微积分的回忆，并提供了一个很好的API来使用这种方法获得本地反向模式AD检查点，但是我对目前为止的进展情况感到非常满意。

The closest-related implementation in Haskell I have seen is the forward mode at http://hackage.haskell.org/packages/archive/fad/1.0/doc/html/Numeric-FAD.html.

The closest related related research appears to be reverse mode for another functional language related to Scheme at http://www.bcl.hamilton.ie/~qobi/stalingrad/.

I see reverse mode in Haskell as kind of a holy grail for a lot of tasks, with the hopes that it could use Haskell's nested data parallelism to gain a nice speedup in heavy numerical optimization.

解决方案

In response to this question, I've uploaded a package named ad to Hackage for handling reverse-mode automatic differentiation in Haskell.

Internally, it leverages a trick from Andy Gill's Kansas Lava to observe sharing in the tape it records for back propagation purposes, and uses type level branding to avoid confusing sensitivities.

I've tried to keep the API relatively close to that of Barak Pearlmutter and Jeffrey Mark Siskind's fad package, but I couldn't resist making a couple of minor tweaks here and there for generality.

I still need to go through and finish up the remaining unimplemented fad combinators, figure out a nice way to build a reverse-mode AD tower, validate that I didn't screw up my recollection of basic calculus, and provide a nice API for using this approach to get local reverse mode checkpoints in an otherwise forward mode AD program, but I am quite happy with how things have progressed thus far.

这篇关于Haskell有反向模式自动区分的实现吗？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！