Julia DifferentialEquations.jl 速度 [英] Julia DifferentialEquations.jl speed

问题描述

我正在尝试测试 Julia ODE 求解器的速度.我在教程中使用了洛伦兹方程:

I am trying to test the speed of Julia ODE solvers. I used the Lorenz equation in the tutorial:

using DifferentialEquations
using Plots
function lorenz(t,u,du)
du[1] = 10.0*(u[2]-u[1])
du[2] = u[1]*(28.0-u[3]) - u[2]
du[3] = u[1]*u[2] - (8/3)*u[3]
end

u0 = [1.0;1.0;1.0]
tspan = (0.0,100.0)
prob = ODEProblem(lorenz,u0,tspan)
sol = solve(prob,reltol=1e-8,abstol=1e-8,saveat=collect(0:0.01:100))

一开始加载包大约需要 25 秒，而代码在 Jupyter 笔记本的 Windows 10 四核笔记本电脑上运行了 7 秒.我知道 Julia 需要先预编译包，这就是加载时间这么长的原因吗?我发现 25 秒无法忍受.此外，当我使用不同的初始值再次运行求解器时，运行时间(~1s)要少得多，这是为什么呢?这是典型的速度吗?

Loading the packages took about 25 s in the beginning, and the code ran for 7 s on a windows 10 quad core laptop in Jupyter notebook. I understand that Julia need to precompile packages first, and is that why the loading time was so long? I found 25 s unbearable. Also, when I ran the solver again using different initial values it took much less time (~1s) to run, and why is that? Is this the typical speed?

推荐答案

Tl;dr:

1. Julia 包有一个预编译阶段.这有助于使所有进一步的 using 调用更快，但第一个调用会存储一些编译数据.这仅在每次包更新时触发.
2. using 必须拉入包，这需要一点时间(取决于可以预编译多少).
3. 预编译不是完整的"，所以当你第一次运行一个函数时，即使是从一个包中，它也必须编译.
4. Julia 开发人员知道这一点，并且已经计划通过使预编译更加完整来摆脱 (2) 和 (3).还有计划减少编译时间，我不知道具体细节.
5. 所有 Julia 函数都专门针对给定的类型，并且每个函数都是一个单独的类型，因此 DiffEq 的内部函数专门针对您给定的每个 ODE 函数.
6. 在大多数需要长时间计算的情况下，(5) 实际上并不重要，因为您不会经常更改函数(如果是，请考虑更改参数).
7. 但是 (6) 在交互式使用时确实很重要.它让人感觉不那么流畅".
8. 我们可以摆脱 ODE 函数的这种特殊化，但它不是默认设置，因为它会导致 2 到 4 倍的性能命中.也许将来会成为默认设置.
9. 我们在这个问题上的预编译时间仍然比 SciPy 封装的 Fortran 求解器在解决此类问题上的时间要好 20 倍.所以这都是编译时问题，而不是运行时问题.编译时间基本上是恒定的(调用相同函数的较大问题具有大致相同的编译)，因此这实际上只是一个交互性问题.
10. 我们(以及整个 Julia)在未来可以并且会在交互方面做得更好.

1. Julia packages have a precompilation phase. This helps make all further using calls quicker, at the cost of the first one storing some compilation data. This is only triggered each package update.
2. using has to pull in the package which takes a little bit (dependent on how much can precompile).
3. Precompilation isn't "complete", so the first time you run a function, even from a package, it will have to compile.
4. Julia devs know about this and there's already plans to get rid of (2) and (3) by making precompilation more complete. There's also plans to reduce compilation time, which I don't know details about.
5. All Julia functions specialize on the types that are given, and each function is a separate type, so DiffEq's internal functions are specializing on each ODE function you give.
6. In most cases with long computations, (5) doesn't actually matter since you aren't changing functions that often (if you are, consider changing parameters instead).
7. But (6) does matter when using it interactively. It makes it feel less "smooth".
8. We can get rid of this specialization on the ODE function, but it isn't the default because it causes a 2x-4x performant hit. Maybe it will be the default in the future.
9. Our timings post precompilation on this problem are still better than things like SciPy's wrapped Fortran solvers on problems like this by 20x. So this is all a compilation time problem, and not a runtime problem. Compilation time is essentially constant (larger problems calling the same function have about the same compilation), so this is really just an interactivity problem.
10. We (and Julia in general) can and will do better with interactivity in the future.

完整说明

这真的不是一个DifferentialEquations.jl 的东西，这只是一个Julia 包的东西.25s 必须包括预编译时间.第一次加载 Julia 包时，它会进行预编译.然后在下一次更新之前不需要再次发生这种情况.这可能是最长的初始化时间，而且对于DifferentialEquations.jl 来说也很长，但同样只有在您每次更新包代码时才会发生这种情况.然后，每次 使用 都会产生少量的初始化成本.DiffEq 相当大，所以初始化需要一点时间:

Full Explanation

This really isn't a DifferentialEquations.jl thing, this is just a Julia package thing. 25s would have to be including the precompilation time. The first time you load a Julia package it precompiles. Then that doesn't need to happen again until the next update. That's probably the longest initialization and it is quite long for DifferentialEquations.jl, but again that only happens each time you update the package code. Then, each time there's a small initialization cost for using. DiffEq is quite large, so it does take a bit to initialize:

@time using DifferentialEquations
5.201393 seconds (4.16 M allocations: 235.883 MiB, 4.09% gc time)

然后，正如您在评论中所指出的那样:

Then as noted in the comments you also have:

@time using Plots
6.499214 seconds (2.48 M allocations: 140.948 MiB, 0.74% gc time)

然后，第一次运行

function lorenz(t,u,du)
  du[1] = 10.0*(u[2]-u[1])
  du[2] = u[1]*(28.0-u[3]) - u[2]
  du[3] = u[1]*u[2] - (8/3)*u[3]
end

u0 = [1.0;1.0;1.0]
tspan = (0.0,100.0)
prob = ODEProblem(lorenz,u0,tspan)
@time sol = solve(prob,reltol=1e-8,abstol=1e-8,saveat=collect(0:0.01:100))

6.993946 seconds (7.93 M allocations: 436.847 MiB, 1.47% gc time)

但是第二次和第三次:

0.010717 seconds (72.21 k allocations: 6.904 MiB)
0.011703 seconds (72.21 k allocations: 6.904 MiB)

那么这里发生了什么?Julia 第一次运行一个函数时，它会编译它.所以当你第一次运行 solve 时，它会在运行时编译所有的内部函数.所有进行的时间都将没有编译.DifferentialEquations.jl 也专门研究函数本身，所以如果我们改变函数:

So what's going on here? The first time Julia runs a function, it will compile it. So the first time you run solve, it will compile all of its internal functions as it runs. All of the proceeding times will be without the compilation. DifferentialEquations.jl also specializes on the function itself, so if we change the function:

function lorenz2(t,u,du)
  du[1] = 10.0*(u[2]-u[1])
  du[2] = u[1]*(28.0-u[3]) - u[2]
  du[3] = u[1]*u[2] - (8/3)*u[3]
end

u0 = [1.0;1.0;1.0]
tspan = (0.0,100.0)
prob = ODEProblem(lorenz2,u0,tspan)

我们将再次产生一些编译时间:

we will incur some of the compilation time again:

@time sol = 
solve(prob,reltol=1e-8,abstol=1e-8,saveat=collect(0:0.01:100))
3.690755 seconds (4.36 M allocations: 239.806 MiB, 1.47% gc time)

原来是这样，现在是为什么.这里有几件事.首先，Julia 包没有完全预编译.它们不会在会话之间保留实际方法的缓存编译版本.这是 1.x 版本列表中要做的事情，这将摆脱第一次命中，类似于仅调用 C/Fortran 包，因为它会提前大量编译 (AOT)职能.所以这很好，但现在只需注意有一个启动时间.

So that's the what, now the why. There's a few things together here. First of all, Julia packages do not fully precompile. They don't keep the cached compiled versions of actual methods between sessions. This is something that is on the 1.x release list to do, and this would get rid of that first hit, similar to just calling a C/Fortran package since it would just be hitting a lot of ahead of time (AOT) compiled functions. So that'll be nice, but for now just note that there is a startup time.

现在让我们谈谈更改功能.Julia 中的每个函数都会自动专门处理其参数(有关详细信息，请参阅此博客文章).这里的关键思想是 Julia 中的每个函数都是一个单独的具体类型.所以，由于这里的问题类型是参数化的，改变函数会触发编译.请注意，就是这种关系:您可以更改函数的参数(如果您有参数)，您可以更改初始条件等，但它只是更改触发重新编译的类型.

Now let's talk about changing the functions. Every function in Julia automatically specializes on its arguments (see this blog post for details). The key idea here is that every function in Julia is a separate concrete type. So, since the problem type here is parameterized, changing the function triggers compilation. Note it's that relation: you can change parameters of the function (if you had parameters), you can change the initial conditions, etc., but it's only changing the type that triggers recompilation.

值得吗?也许.我们希望专门为难以计算的东西快速处理.编译时间是恒定的(即你可以求解一个 6 小时的 ODE，它仍然需要几秒钟)，因此计算成本高的计算不会在这里受到影响.在这里运行数千个参数和初始条件的蒙特卡罗模拟不会受到影响，因为如果您只是更改初始条件和参数的值，那么它将不会重新编译.但是，在您更改功能的地方进行交互使用确实会受到第二次左右的影响，这并不好.Julia 开发人员对此的一个回答是，在 Julia 1.0 之后花费时间来加快编译时间，这是我不知道细节的事情，但我确信这里有一些唾手可得的成果.

Is it worth it? Well, maybe. We want to specialize to have things fast for calculations which are difficult. Compilation time is constant (i.e. you can solve a 6 hour ODE and it'll still be a few seconds), and so the computationally-costly calculations aren't effected here. Monte Carlo simulations where you're running thousands of parameters and initial conditions aren't effected here because if you're just changing values of initial conditions and parameters then it won't recompile. But interactive use where you are changing functions does get a second or so hit in there, which isn't nice. One answer from the Julia devs for this is to spend post Julia 1.0 time speeding up compilation times, which is something that I don't know the details of but I am assured there's some low hanging fruit here.

我们可以摆脱它吗?是的.DiffEq Online 不会为每个函数重新编译，因为它面向在线使用.

Can we get rid of it? Yes. DiffEq Online doesn't recompile for each function because it's geared towards online use.

function lorenz3(t,u,du)
  du[1] = 10.0*(u[2]-u[1])
  du[2] = u[1]*(28.0-u[3]) - u[2]
  du[3] = u[1]*u[2] - (8/3)*u[3]
  nothing
end

u0 = [1.0;1.0;1.0]
tspan = (0.0,100.0)
f = NSODEFunction{true}(lorenz3,tspan[1],u0)
prob = ODEProblem{true}(f,u0,tspan)

@time sol = solve(prob,reltol=1e-8,abstol=1e-8,saveat=collect(0:0.01:100))

1.505591 seconds (860.21 k allocations: 38.605 MiB, 0.95% gc time)

现在我们可以更改函数而不产生编译成本:

And now we can change the function and not incur compilation cost:

function lorenz4(t,u,du)
  du[1] = 10.0*(u[2]-u[1])
  du[2] = u[1]*(28.0-u[3]) - u[2]
  du[3] = u[1]*u[2] - (8/3)*u[3]
  nothing
end

u0 = [1.0;1.0;1.0]
tspan = (0.0,100.0)
f = NSODEFunction{true}(lorenz4,tspan[1],u0)
prob = ODEProblem{true}(f,u0,tspan)

@time sol = 
solve(prob,reltol=1e-8,abstol=1e-8,saveat=collect(0:0.01
:100))

0.038276 seconds (242.31 k allocations: 10.797 MiB, 22.50% gc time)

而 tada，通过将函数包装在 NSODEFunction 中(内部使用 FunctionWrappers.jl)它不再专门针对每个函数，并且您在每个 Julia 会话中点击一次编译时间(然后一旦缓存，每次包更新一次).但请注意，这大约有 2x-4x 成本 所以我不确定它是否会默认启用.我们可以在问题类型构造函数中默认实现这一点(即默认情况下没有额外的专业化，但用户可以以交互性为代价选择更快的速度)但我不确定这里更好的默认值是什么(随意用你的想法评论这个问题).但它肯定会在 Julia 更改其关键字参数并因此免编译"之后很快被记录下来.模式将是使用它的标准方式，即使不是默认方式.

And tada, by wrapping the function in NSODEFunction (which is internally using FunctionWrappers.jl) it no longer specializes per-function and you hit the compilation time once per Julia session (and then once that's cached, once per package update). But notice that this has about a 2x-4x cost so I am not sure if it will be enabled by default. We could make this happen by default inside of the problem-type constructor (i.e. no extra specialization by default, but the user can opt into more speed at the cost of interactivity) but I am unsure what the better default is here (feel free to comment on the issue with your thoughts). But it will definitely get documented soon after Julia does its keyword argument changes and so "compilation-free" mode will be a standard way to use it, even if not default.

但只是为了把它放在透视图中，

But just to put it into perspective,

import numpy as np
from scipy.integrate import odeint
y0 = [1.0,1.0,1.0]
t = np.linspace(0, 100, 10001)
def f(u,t):
    return [10.0*(u[1]-u[0]),u[0]*(28.0-u[2])-u[1],u[0]*u[1]-(8/3)*u[2]]
%timeit odeint(f,y0,t,atol=1e-8,rtol=1e-8)

1 loop, best of 3: 210 ms per loop

我们正在研究是否应该将这种交互便利性默认设置为比此处 SciPy 的默认设置快 5 倍而不是 20 倍(尽管我们的默认设置通常比 SciPy 使用的默认设置更准确，但这是另一个数据时间可以在基准中找到或只是询问).一方面它易于使用是有意义的，但另一方面，如果重新启用长计算的专业化并且不知道蒙特卡洛(这是你真正想要速度的地方)，那么那里的很多人会受到 2 到 4 倍的性能影响，这可能需要额外的几天/几周的计算.呃……艰难的选择.

we're looking at whether this interactive convenience should be made a default to be 5x faster instead of 20x faster than SciPy's default here (though our default will usually be much more accurate than the default SciPy uses, but that's data for another time which can be found in the benchmarks or just ask). On one hand it makes sense as ease-of-use, but on the other hand if re-enabling the specialization for long calculations and Monte Carlo isn't known (which is where you really want speed), then lots of people there will take a 2x-4x performance hit which could amount to extra days/weeks of computation. Ehh... tough choices.

因此，最终混合了优化选择和 Julia 缺少的一些预编译功能，这些功能会影响交互性而不会影响真正的运行时速度.如果您希望使用一些大的 Monte Carlo 来估计参数，或者解决大量的 SDE，或者解决一个大的 PDE，我们可以解决这个问题.那是我们的第一个目标，我们确保尽可能地达到目标.但是在 REPL 中玩耍确实有 2-3 秒的故障".我们也不能忽视这一点(当然比在 C/Fortran 中玩耍要好，但对于 REPL 来说仍然不理想).为此，我已经向您展示了已经在开发和测试的解决方案，因此希望明年的这个时候我们可以针对特定案例提供更好的答案.

So in the end there's a mixture of optimizing choices and some precompilation features missing from Julia that effect the interactivity without effecting the true runtime speed. If you're looking to estimate parameters using some big Monte Carlo, or solve a ton of SDEs, or solve a big PDE, we have that down. That was our first goal and we made sure to hit that as good as possible. But playing around in the REPL does have 2-3 second "gliches" which we also cannot ignore (better than playing around in C/Fortran though of course, but still not ideal for a REPL). For this, I've shown you that there's solutions already being developed and tested, and so hopefully this time next year we can have a better answer for that specific case.

另外两点需要注意.如果您只使用 ODE 求解器，您可以执行 using OrdinaryDiffEq 以继续下载/安装/编译/导入所有 DifferentialEquations.jl (这在手册中有描述).此外，像这样使用 saveat 可能不是解决这个问题的最快方法:用更少的点来解决它并在必要时使用密集输出可能会更好.

Two other things to note. If you're only using the ODE solvers, you can just do using OrdinaryDiffEq to keep downloading/installing/compiling/importing all of DifferentialEquations.jl (this is described in the manual). Also, using saveat like that probably isn't the fastest way to solve this problem: solving it with a lot less points and using the dense output as necessary may be better here.

我打开了一个问题，详细说明了我们如何减少函数之间"编译时间而不会失去专业化提供的加速.我认为这是我们可以在短期内优先考虑的事情，因为我同意我们可以在这里做得更好.

I opened an issue detailing how we can reduce the "between function" compilation time without losing the speedup that specializing gives. I think this is something we can make a short-term priority since I agree that we could do better here.

这篇关于Julia DifferentialEquations.jl 速度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！