如何在 Julia 中进行正确的微基准测试? [英] How do I do a correct micro-benchmark in Julia?

问题描述

Julia 1.0.0 documentation 提供一般提示.

The Julia 1.0.0 documentation provides general tips.

它还建议不要使用@time 宏:

It also suggests that instead of using the @time macro:

对于更严格的基准测试，请考虑 BenchmarkTools.jl 包，其中包括多次评估函数以减少噪音.

For more serious benchmarking, consider the BenchmarkTools.jl package which among other things evaluates the function multiple times in order to reduce noise.

它们在使用方面如何比较，使用基础"Julia 之外的东西是否值得?

How do they compare in use and is it worth the trouble to use something not in "base" Julia?

推荐答案

从统计上看，@benchmark 比@time 好很多

TL;DR BenchmarkTools @benchmark 宏是一个很棒的微基准测试工具.谨慎使用 @time 宏，不要认真对待第一次运行.

TL;DR The BenchmarkTools @benchmark macro is a great micro-benchmark tool. Use the @time macro with caution and don't take the first run seriously.

这个简单的例子说明了用途和区别:

This simple example illustrates use and differences:

julia> # Fresh Julia 1.0.0 REPL

julia> # Add BenchmarkTools package using ] key package manager

(v1.0) pkg> add BenchmarkTools  
julia> # Press backspace key to get back to Julia REPL

# Load BenchmarkTools package into current REPL
julia> using BenchmarkTools

julia> # Definine a function with a known elapsed time
julia> f(n) = sleep(n)  # n is in seconds
f (generic function with 1 method)

# Expect just over 500 ms for elapsed time
julia> @benchmark f(0.5)
BenchmarkTools.Trial:
  memory estimate:  192 bytes
  allocs estimate:  5
  --------------
  minimum time:     501.825 ms (0.00% GC)
  median time:      507.386 ms (0.00% GC)
  mean time:        508.069 ms (0.00% GC)
  maximum time:     514.496 ms (0.00% GC)
  --------------
  samples:          10
  evals/sample:     1

julia> # Try second run to compare consistency
julia> # Note the very close consistency in ms for both median and mean times

julia> @benchmark f(0.5)
BenchmarkTools.Trial:
  memory estimate:  192 bytes
  allocs estimate:  5
  --------------
  minimum time:     502.603 ms (0.00% GC)
  median time:      508.716 ms (0.00% GC)
  mean time:        508.619 ms (0.00% GC)
  maximum time:     515.602 ms (0.00% GC)
  --------------
  samples:          10
  evals/sample:     1


julia> # Define the same function with new name for @time macro tests
julia> g(n) = sleep(n)
g (generic function with 1 method)

# First run suffers from compilation time, so 518 ms
julia> @time sleep(0.5)
  0.517897 seconds (83 allocations: 5.813 KiB)

# Second run drops to 502 ms, 16 ms drop
julia> @time sleep(0.5)
  0.502038 seconds (9 allocations: 352 bytes)

# Third run similar to second
julia> @time sleep(0.5)
  0.503606 seconds (9 allocations: 352 bytes)

# Fourth run increases over second by about 13 ms
julia> @time sleep(0.5)
  0.514629 seconds (9 allocations: 352 bytes)

这个简单的例子说明了使用 @benchmark 宏是多么容易，以及应该谨慎对待 @time 宏结果.

This simple example illustrates how easy it is to use the @benchmark macro and the caution with which the @time macro results should be taken.

是的，使用 @benchmark 宏是值得的.

Yes, it is worth the trouble to use the @benchmark macro.

这篇关于如何在 Julia 中进行正确的微基准测试?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！