朱莉娅微分方程.jl速度 [英] Julia DifferentialEquations.jl speed

问题描述

我正在尝试测试Julia ODE求解器的速度.我在教程中使用了Lorenz方程:

I am trying to test the speed of Julia ODE solvers. I used the Lorenz equation in the tutorial:

using DifferentialEquations
using Plots
function lorenz(t,u,du)
du[1] = 10.0*(u[2]-u[1])
du[2] = u[1]*(28.0-u[3]) - u[2]
du[3] = u[1]*u[2] - (8/3)*u[3]
end

u0 = [1.0;1.0;1.0]
tspan = (0.0,100.0)
prob = ODEProblem(lorenz,u0,tspan)
sol = solve(prob,reltol=1e-8,abstol=1e-8,saveat=collect(0:0.01:100))

开始加载软件包大约需要25秒，并且代码在Jupyter笔记本的Windows 10四核笔记本电脑上运行了7秒钟.我了解到Julia需要首先预编译软件包，这就是为什么加载时间这么长吗?我发现25 s难以忍受.另外，当我使用不同的初始值再次运行求解器时，它花费的时间要少得多(〜1秒)，为什么?这是典型的速度吗?

Loading the packages took about 25 s in the beginning, and the code ran for 7 s on a windows 10 quad core laptop in Jupyter notebook. I understand that Julia need to precompile packages first, and is that why the loading time was so long? I found 25 s unbearable. Also, when I ran the solver again using different initial values it took much less time (~1s) to run, and why is that? Is this the typical speed?

推荐答案

Tl; dr:

1. Julia软件包具有预编译阶段.这有助于使所有进一步的 using 调用更快，而第一个调用会存储一些编译数据.这仅在每次软件包更新时触发.
2. 使用必须提取需要一点点的软件包(取决于可以预编译的数量).
3. 预编译不是完整的"，因此，即使您是从程序包中首次运行函数，也必须对其进行编译.
4. Julia开发人员对此很了解，并且已经有计划通过使预编译更加完整来摆脱(2)和(3).还计划减少编译时间，我不知道这方面的详细信息.
5. 所有Julia函数都专门研究给定的类型，每个函数都是单独的类型，因此DiffEq的内部函数专门研究您提供的每个ODE函数.
6. 在大多数情况下，计算量很大，(5)实际上并不重要，因为您不经常更改函数(如果需要的话，请考虑更改参数).
7. 但是(6)交互使用时确实很重要.它使人感觉不那么光滑".
8. 我们可以摆脱ODE函数的这种专业化，但这不是默认设置，因为它会导致2到4倍的性能冲击.也许它将是将来的默认设置.
9. 我们在此问题上进行预编译后的时机仍然比SciPy封装的Fortran求解器要好20倍之类的东西要好.因此，这全是编译时的问题，而不是运行时的问题.编译时间本质上是恒定的(调用同一函数的较大问题大约具有相同的编译)，因此这实际上只是一个交互性问题.
10. 我们(以及一般来说，朱莉娅)将来可以而且将会在互动方面做得更好.

1. Julia packages have a precompilation phase. This helps make all further using calls quicker, at the cost of the first one storing some compilation data. This is only triggered each package update.
2. using has to pull in the package which takes a little bit (dependent on how much can precompile).
3. Precompilation isn't "complete", so the first time you run a function, even from a package, it will have to compile.
4. Julia devs know about this and there's already plans to get rid of (2) and (3) by making precompilation more complete. There's also plans to reduce compilation time, which I don't know details about.
5. All Julia functions specialize on the types that are given, and each function is a separate type, so DiffEq's internal functions are specializing on each ODE function you give.
6. In most cases with long computations, (5) doesn't actually matter since you aren't changing functions that often (if you are, consider changing parameters instead).
7. But (6) does matter when using it interactively. It makes it feel less "smooth".
8. We can get rid of this specialization on the ODE function, but it isn't the default because it causes a 2x-4x performant hit. Maybe it will be the default in the future.
9. Our timings post precompilation on this problem are still better than things like SciPy's wrapped Fortran solvers on problems like this by 20x. So this is all a compilation time problem, and not a runtime problem. Compilation time is essentially constant (larger problems calling the same function have about the same compilation), so this is really just an interactivity problem.
10. We (and Julia in general) can and will do better with interactivity in the future.

完整说明

这确实不是DifferentialEquations.jl，这只是Julia包中的东西.25s必须包括预编译时间.首次加载Julia程序包时，它将进行预编译.然后，无需再次进行此操作，直到下一次更新.这可能是最长的初始化，并且对DifferentialEquations.jl来说是相当长的时间，但是同样，这仅在每次更新程序包代码时才会发生.然后，每次使用 using 都会花费很少的初始化费用.DiffEq很大，因此初始化需要一点时间:

Full Explanation

This really isn't a DifferentialEquations.jl thing, this is just a Julia package thing. 25s would have to be including the precompilation time. The first time you load a Julia package it precompiles. Then that doesn't need to happen again until the next update. That's probably the longest initialization and it is quite long for DifferentialEquations.jl, but again that only happens each time you update the package code. Then, each time there's a small initialization cost for using. DiffEq is quite large, so it does take a bit to initialize:

@time using DifferentialEquations
5.201393 seconds (4.16 M allocations: 235.883 MiB, 4.09% gc time)

然后，如注释中所述，您还拥有:

Then as noted in the comments you also have:

@time using Plots
6.499214 seconds (2.48 M allocations: 140.948 MiB, 0.74% gc time)

然后，第一次运行

function lorenz(t,u,du)
  du[1] = 10.0*(u[2]-u[1])
  du[2] = u[1]*(28.0-u[3]) - u[2]
  du[3] = u[1]*u[2] - (8/3)*u[3]
end

u0 = [1.0;1.0;1.0]
tspan = (0.0,100.0)
prob = ODEProblem(lorenz,u0,tspan)
@time sol = solve(prob,reltol=1e-8,abstol=1e-8,saveat=collect(0:0.01:100))

6.993946 seconds (7.93 M allocations: 436.847 MiB, 1.47% gc time)

但是第二次和第三次:

0.010717 seconds (72.21 k allocations: 6.904 MiB)
0.011703 seconds (72.21 k allocations: 6.904 MiB)

那么这是怎么回事?Julia第一次运行函数时，它将对其进行编译.因此，第一次运行 solve 时，它将在运行时编译其所有内部函数.所有进行时间都将不进行编译.DifferentialEquations.jl还专门研究函数本身，因此，如果我们更改函数，则:

So what's going on here? The first time Julia runs a function, it will compile it. So the first time you run solve, it will compile all of its internal functions as it runs. All of the proceeding times will be without the compilation. DifferentialEquations.jl also specializes on the function itself, so if we change the function:

function lorenz2(t,u,du)
  du[1] = 10.0*(u[2]-u[1])
  du[2] = u[1]*(28.0-u[3]) - u[2]
  du[3] = u[1]*u[2] - (8/3)*u[3]
end

u0 = [1.0;1.0;1.0]
tspan = (0.0,100.0)
prob = ODEProblem(lorenz2,u0,tspan)

我们将再次产生一些编译时间:

we will incur some of the compilation time again:

@time sol = 
solve(prob,reltol=1e-8,abstol=1e-8,saveat=collect(0:0.01:100))
3.690755 seconds (4.36 M allocations: 239.806 MiB, 1.47% gc time)

这就是原因，现在是原因.这里有几件事.首先，Julia程序包不能完全预编译.它们不会在会话之间保留实际方法的已缓存编译版本.这是1.x版本列表中要执行的操作，它将摆脱第一个影响，类似于仅调用C/Fortran程序包，因为它将提前很多时间(AOT)进行编译职能.这样很好，但是现在请注意，有一个启动时间.

So that's the what, now the why. There's a few things together here. First of all, Julia packages do not fully precompile. They don't keep the cached compiled versions of actual methods between sessions. This is something that is on the 1.x release list to do, and this would get rid of that first hit, similar to just calling a C/Fortran package since it would just be hitting a lot of ahead of time (AOT) compiled functions. So that'll be nice, but for now just note that there is a startup time.

现在让我们谈谈更改功能.Julia中的每个函数都会自动专注于其参数(有关详细信息，请参见此博客文章).这里的关键思想是Julia中的每个函数都是单独的具体类型.因此，由于此处的问题类型已参数化，因此更改函数会触发编译.请注意就是这种关系:您可以更改函数的参数(如果有参数)，可以更改初始条件等，但这只是更改触发重新编译的类型.

Now let's talk about changing the functions. Every function in Julia automatically specializes on its arguments (see this blog post for details). The key idea here is that every function in Julia is a separate concrete type. So, since the problem type here is parameterized, changing the function triggers compilation. Note it's that relation: you can change parameters of the function (if you had parameters), you can change the initial conditions, etc., but it's only changing the type that triggers recompilation.

值得吗?也许.我们想专门进行快速处理困难的事情.编译时间是恒定的(即，您可以求解6小时的ODE，并且仍然需要几秒钟的时间)，因此此处不会进行计算量大的计算.在此处运行数千个参数和初始条件的Monte Carlo模拟不会受到影响，因为如果您仅更改初始条件和参数的值，则它将不会重新编译.但是在要更改功能的地方进行交互使用确实能获得一秒钟左右的效果，但这并不好.Julia开发者对此的一个答案是花费Julia 1.0以后的时间来加快编译时间，这是我不知道的细节，但是我确信这里会有一些悬而未决的成果.

Is it worth it? Well, maybe. We want to specialize to have things fast for calculations which are difficult. Compilation time is constant (i.e. you can solve a 6 hour ODE and it'll still be a few seconds), and so the computationally-costly calculations aren't effected here. Monte Carlo simulations where you're running thousands of parameters and initial conditions aren't effected here because if you're just changing values of initial conditions and parameters then it won't recompile. But interactive use where you are changing functions does get a second or so hit in there, which isn't nice. One answer from the Julia devs for this is to spend post Julia 1.0 time speeding up compilation times, which is something that I don't know the details of but I am assured there's some low hanging fruit here.

我们可以摆脱它吗?是的. DiffEq Online 不会针对每个函数重新编译，因为它是为在线使用而设计的.

Can we get rid of it? Yes. DiffEq Online doesn't recompile for each function because it's geared towards online use.

function lorenz3(t,u,du)
  du[1] = 10.0*(u[2]-u[1])
  du[2] = u[1]*(28.0-u[3]) - u[2]
  du[3] = u[1]*u[2] - (8/3)*u[3]
  nothing
end

u0 = [1.0;1.0;1.0]
tspan = (0.0,100.0)
f = NSODEFunction{true}(lorenz3,tspan[1],u0)
prob = ODEProblem{true}(f,u0,tspan)

@time sol = solve(prob,reltol=1e-8,abstol=1e-8,saveat=collect(0:0.01:100))

1.505591 seconds (860.21 k allocations: 38.605 MiB, 0.95% gc time)

现在我们可以更改功能，而不会产生编译成本:

And now we can change the function and not incur compilation cost:

function lorenz4(t,u,du)
  du[1] = 10.0*(u[2]-u[1])
  du[2] = u[1]*(28.0-u[3]) - u[2]
  du[3] = u[1]*u[2] - (8/3)*u[3]
  nothing
end

u0 = [1.0;1.0;1.0]
tspan = (0.0,100.0)
f = NSODEFunction{true}(lorenz4,tspan[1],u0)
prob = ODEProblem{true}(f,u0,tspan)

@time sol = 
solve(prob,reltol=1e-8,abstol=1e-8,saveat=collect(0:0.01
:100))

0.038276 seconds (242.31 k allocations: 10.797 MiB, 22.50% gc time)

然后是tada，将函数包装在 NSODEFunction 中(内部使用FunctionWrappers.jl )，它不再专门针对每个功能，您会在每个Julia会话中遇到一次编译时间(然后缓存一次，每次软件包更新一次).但是请注意，这大约需要2到4倍的成本所以我不确定是否会默认情况下启用.我们可以默认在问题类型构造函数内部实现此目的(即默认情况下无需额外的专业化，但用户可以以交互性为代价选择更高的速度)，但是我不确定这里有更好的默认值(请随意用您的想法对问题发表评论).但是在Julia更改其关键字参数并因此无编译"后，它肯定会很快被记录在案.模式是使用它的一种标准方式，即使不是默认方式也是如此.

And tada, by wrapping the function in NSODEFunction (which is internally using FunctionWrappers.jl) it no longer specializes per-function and you hit the compilation time once per Julia session (and then once that's cached, once per package update). But notice that this has about a 2x-4x cost so I am not sure if it will be enabled by default. We could make this happen by default inside of the problem-type constructor (i.e. no extra specialization by default, but the user can opt into more speed at the cost of interactivity) but I am unsure what the better default is here (feel free to comment on the issue with your thoughts). But it will definitely get documented soon after Julia does its keyword argument changes and so "compilation-free" mode will be a standard way to use it, even if not default.

但只是为了说明问题，

import numpy as np
from scipy.integrate import odeint
y0 = [1.0,1.0,1.0]
t = np.linspace(0, 100, 10001)
def f(u,t):
    return [10.0*(u[1]-u[0]),u[0]*(28.0-u[2])-u[1],u[0]*u[1]-(8/3)*u[2]]
%timeit odeint(f,y0,t,atol=1e-8,rtol=1e-8)

1 loop, best of 3: 210 ms per loop

我们正在研究是否应将此交互便利性的默认值设置为比SciPy的默认设置快5倍，而不是20倍(尽管我们的默认设置通常比SciPy所使用的默认设置准确得多，但这是另一个方面的数据)时间可以在基准测试中找到，也可以问一下).一方面，这很容易使用，但另一方面，如果重新启用长时间计算的专业化功能，而蒙特卡洛不为人所知(这是您真正想要的速度)，那么那里的很多人都会造成2到4倍的性能下降，这可能会导致额外的天数/周的计算量.嗯...艰难的选择.

we're looking at whether this interactive convenience should be made a default to be 5x faster instead of 20x faster than SciPy's default here (though our default will usually be much more accurate than the default SciPy uses, but that's data for another time which can be found in the benchmarks or just ask). On one hand it makes sense as ease-of-use, but on the other hand if re-enabling the specialization for long calculations and Monte Carlo isn't known (which is where you really want speed), then lots of people there will take a 2x-4x performance hit which could amount to extra days/weeks of computation. Ehh... tough choices.

因此，最终，Julia缺少了优化选择和一些预编译功能的混合，这些功能会影响交互性而不会影响真正的运行时速度.如果您想使用一些较大的Monte Carlo来估计参数，或者要解决大量的SDE，或者要解决较大的PDE，那么我们不建议这样做.那是我们的第一个目标，我们确保做到最好.但是在REPL中玩耍确实有2-3秒的故障".我们也不能忽略它(当然比在C/Fortran中玩得更好，但是对于REPL来说仍然不是理想选择).为此，我向您展示了已经开发和测试了解决方案，因此希望明年次我们可以针对该特定情况提供更好的答案.

So in the end there's a mixture of optimizing choices and some precompilation features missing from Julia that effect the interactivity without effecting the true runtime speed. If you're looking to estimate parameters using some big Monte Carlo, or solve a ton of SDEs, or solve a big PDE, we have that down. That was our first goal and we made sure to hit that as good as possible. But playing around in the REPL does have 2-3 second "gliches" which we also cannot ignore (better than playing around in C/Fortran though of course, but still not ideal for a REPL). For this, I've shown you that there's solutions already being developed and tested, and so hopefully this time next year we can have a better answer for that specific case.

另外两件事要注意.如果仅使用ODE求解器，则可以使用OrdinaryDiffEq 进行操作，以继续下载/安装/编译/导入所有DifferentialEquations.jl(

Two other things to note. If you're only using the ODE solvers, you can just do using OrdinaryDiffEq to keep downloading/installing/compiling/importing all of DifferentialEquations.jl (this is described in the manual). Also, using saveat like that probably isn't the fastest way to solve this problem: solving it with a lot less points and using the dense output as necessary may be better here.

我打开了一个问题，详细说明了如何减少函数之间"编译时间而又不会失去专业化所带来的加速效果.我认为这是我们可以短期优先考虑的事情，因为我同意我们可以在这里做得更好.

I opened an issue detailing how we can reduce the "between function" compilation time without losing the speedup that specializing gives. I think this is something we can make a short-term priority since I agree that we could do better here.

这篇关于朱莉娅微分方程.jl速度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！