如何在Julia中进行正确的微基准测试? [英] How do I do a correct micro-benchmark in Julia?

问题描述

Julia 1.0.0

The Julia 1.0.0 documentation provides general tips.

它还建议您不要使用@time宏:

It also suggests that instead of using the @time macro:

要进行更严格的基准测试，请考虑BenchmarkTools.jl软件包，该软件包除其他功能外还多次评估该功能，以降低噪声.

For more serious benchmarking, consider the BenchmarkTools.jl package which among other things evaluates the function multiple times in order to reduce noise.

它们在使用中如何进行比较?使用不在基本" Julia中的东西值得麻烦吗?

How do they compare in use and is it worth the trouble to use something not in "base" Julia?

推荐答案

从统计角度来看，@ benchmark比@time好得多

TL; DR BenchmarkTools @benchmark宏是一个很棒的微基准测试工具. 小心使用@time宏，不要认真对待第一次运行.

TL;DR The BenchmarkTools @benchmark macro is a great micro-benchmark tool. Use the @time macro with caution and don't take the first run seriously.

这个简单的例子说明了用法和区别:

This simple example illustrates use and differences:

julia> # Fresh Julia 1.0.0 REPL

julia> # Add BenchmarkTools package using ] key package manager

(v1.0) pkg> add BenchmarkTools  
julia> # Press backspace key to get back to Julia REPL

# Load BenchmarkTools package into current REPL
julia> using BenchmarkTools

julia> # Definine a function with a known elapsed time
julia> f(n) = sleep(n)  # n is in seconds
f (generic function with 1 method)

# Expect just over 500 ms for elapsed time
julia> @benchmark f(0.5)
BenchmarkTools.Trial:
  memory estimate:  192 bytes
  allocs estimate:  5
  --------------
  minimum time:     501.825 ms (0.00% GC)
  median time:      507.386 ms (0.00% GC)
  mean time:        508.069 ms (0.00% GC)
  maximum time:     514.496 ms (0.00% GC)
  --------------
  samples:          10
  evals/sample:     1

julia> # Try second run to compare consistency
julia> # Note the very close consistency in ms for both median and mean times

julia> @benchmark f(0.5)
BenchmarkTools.Trial:
  memory estimate:  192 bytes
  allocs estimate:  5
  --------------
  minimum time:     502.603 ms (0.00% GC)
  median time:      508.716 ms (0.00% GC)
  mean time:        508.619 ms (0.00% GC)
  maximum time:     515.602 ms (0.00% GC)
  --------------
  samples:          10
  evals/sample:     1


julia> # Define the same function with new name for @time macro tests
julia> g(n) = sleep(n)
g (generic function with 1 method)

# First run suffers from compilation time, so 518 ms
julia> @time sleep(0.5)
  0.517897 seconds (83 allocations: 5.813 KiB)

# Second run drops to 502 ms, 16 ms drop
julia> @time sleep(0.5)
  0.502038 seconds (9 allocations: 352 bytes)

# Third run similar to second
julia> @time sleep(0.5)
  0.503606 seconds (9 allocations: 352 bytes)

# Fourth run increases over second by about 13 ms
julia> @time sleep(0.5)
  0.514629 seconds (9 allocations: 352 bytes)

这个简单的示例说明了使用@benchmark宏非常容易，并且应谨慎使用@time宏结果.

This simple example illustrates how easy it is to use the @benchmark macro and the caution with which the @time macro results should be taken.

是的，使用@benchmark宏是值得的麻烦.

Yes, it is worth the trouble to use the @benchmark macro.

这篇关于如何在Julia中进行正确的微基准测试?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！