何时插入基准表达式 [英] When to interpolate in benchmarking expressions

问题描述

BenchmarkTools 文档建议将全局变量插值到基准表达式中.但是，它们提供的示例的运行时间差距似乎已经大大缩小.在他们的示例，它们具有全局变量A = rand(1000)，并且将@benchmark [i*i for i in A]与@benchmark [i*i for i in $A]进行比较，分别得到13.806 μs与1.348 μs.但是，当我现在运行该示例时，运行时间非常接近:

The BenchmarkTools documentation recommends interpolating global variables into benchmarking expressions. However, the gap in run times for the example that they provide seems to have closed considerably. In their example, they have a global variable A = rand(1000), and they compare @benchmark [i*i for i in A] to @benchmark [i*i for i in $A], and get 13.806 μs versus 1.348 μs, respectively. However, when I run that example now, the run times are very close:

julia> using Statistics, BenchmarkTools

julia> A = rand(1000);

julia> median(@benchmark [i*i for i in A])
BenchmarkTools.TrialEstimate: 
  time:             892.821 ns
  gctime:           0.000 ns (0.00%)
  memory:           7.95 KiB
  allocs:           2

julia> median(@benchmark [i*i for i in $A])
BenchmarkTools.TrialEstimate: 
  time:             836.075 ns
  gctime:           0.000 ns (0.00%)
  memory:           7.95 KiB
  allocs:           2

这是我的版本信息:

julia> versioninfo()
Julia Version 1.1.1
Commit 55e36cc (2019-05-16 04:10 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin15.6.0)
  CPU: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

是否仍然需要在基准中进行插值?知道为什么现在的运行时间如此相似吗?任何人都可以提供一个不同的示例，其中运行时间相差远远大于一个吗?

Is interpolation in benchmarks still necessary? Any idea why the run times are so similar now? Can anyone provide a different example where the run times are different by a factor much greater than one?

推荐答案

BenchmarkTools与编译器进行军备竞赛-在多个方面！

BenchmarkTools is in an arms race against the compiler — on multiple fronts!

两个表达式之间的差异等同于这两个函数之间的差异:

The difference between the two expressions is equivalent to the difference between these two functions:

# @benchmark [i*i for i in A]
f1() = [i*i for i in A]

# @benchmark [i*i for i in $A]
f2(X) = [i*i for i in X]

换句话说，使用$会将值视为自变量而不是硬编码的常量或全局值.由于A是全局变量而不是常量，因此f1()是类型不稳定.当然，Julia在处理类型不稳定性方面越来越好，而且看来这是您不再为此付出代价的另一个地方.

In other words, using a $ treats the value as an argument instead of a hard-coded constant or global. Since A is a global but not constant, f1() is type-unstable. Of course Julia has been getting better and better at dealing with type-instabilities and it appears that this is yet another place where you're no longer paying the cost for it.

有些时候，不使用$实际上会获得惊人的快速结果，因为Julia将对值进行硬编码并可能进行某种持续传播，从而使过度专业化精确值您正在进行基准测试.这是一个显示

There are times where not using a $ will actually give deceivingly fast results because Julia will hard-code the value and may do some sort of constant propagation that over-specializes on the exact value you're benchmarking. Here's an example that shows both directions on

julia> x = 0.5; # non-constant global

julia> @btime sin(x);
  20.106 ns (1 allocation: 16 bytes)

julia> @btime sin($x);
  5.413 ns (0 allocations: 0 bytes)

julia> @btime sin(0.5); # constant literal!
  1.818 ns (0 allocations: 0 bytes)

julia> @btime sin($0.5);
  5.416 ns (0 allocations: 0 bytes)

这篇关于何时插入基准表达式的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！