我可以通过哪些方式对 Julia 函数进行基准测试? [英] In what way(s) can I benchmark a Julia function?

问题描述

我自学了机器学习，最近开始深入研究 Julia 机器学习生态系统.<小时>来自 python 背景，有一些 Tensorflow 和 OpenCV/skimage 经验，我想对 Julia ML 进行基准测试库(Flux/JuliaImages)与其对应的比较看看它真正执行CV(任何)任务的快慢，并决定我是否应该转向使用 Julia.

I've self-taught myself machine learning and have recently started delving into the Julia Machine Learning Ecosystem.

---

Coming from a python background and having some Tensorflow and OpenCV/skimage experience, I want to benchmark Julia ML libraries (Flux/JuliaImages) against its counterparts to see how fast or slow it really performs CV(any) task(s) and to decide if I should shift to using Julia.

我知道如何使用 获取在 python 中执行函数所花费的时间timeit 模块像这样:

I know how to get the time taken to execute a function in python using timeit module like this :

#Loading an Image using OpenCV

s = """
img = cv2.imread('sample_image.png', 1)
"""
setup = """
import timeit
"""
print(str(round((timeit.timeit(stmt = s, setup = setup, number = 1))*1000, 2)) + " ms")
#printing the time taken in ms rounded to 2 digits

如何比较使用适当库(在本例中为 JuliaImages)在 Julia 中执行相同任务的函数的执行时间.

How does one compare the execution time of a function performing the same task in Julia using the appropriate library (in this case, JuliaImages).

Julia 是否为时间/基准测试提供任何函数/宏?

Does Julia provide any function/macro to time/benchmark ?

推荐答案

使用 BenchmarkTools 是对 Julia 函数进行基准测试的推荐方法.除非您正在计时需要相当长的时间，否则请使用 @benchmark 或从中导出的不太冗长的 @btime 宏.因为这些宏背后的机制会多次评估目标函数，所以 @time 可用于对运行缓慢的事物进行基准测试(例如，涉及磁盘访问或非常耗时的计算).

using BenchmarkTools is the recommended way to benchmark Julia functions. Unless you are timing something that takes quite a while, use either @benchmark or the less verbose @btime macros exported from it. Because the machinery behind these macros evaluates the target function many times, @time is useful for benchmarking things that run slowly (e.g. where disk access or very time-consuming calculations are involved).

正确使用 @btime 或 @benchmark 很重要，这样可以避免误导结果.通常，您正在对一个接受一个或多个参数的函数进行基准测试.基准测试时，所有参数都应该是外部变量:(没有基准宏)

It is important to use @btime or @benchmark correctly, this avoids misleading results. Usually, you are benchmarking a function that takes one or more arguments. When benchmarking, all arguments should be external variables: (without the benchmark macro)

x = 1
f(x)
# do not use f(1)

函数将被多次评估.为了防止函数参数在每次计算函数时被重新计算，我们必须通过在每个用作参数的变量的名称前加上一个 $ 来标记每个参数.基准测试宏使用它来指示应该在基准测试过程开始时评估(解析)一次变量，然后直接按原样重用结果:

The function will be evaluated many times. To prevent the function arguments from being re-evaluated whenever the function is evaluated, we must mark each argument by prefixing a $ to the name of each variable that is used as an argument. The benchmarking macros use this to indicate that the variable should be evaluated (resolved) once, at the start of the benchmarking process and then the result is to be reused directly as is:

julia> using BenchmarkTools
julia> a = 1/2;
julia> b = 1/4;
julia> c = 1/8;
julia> a, b, c
(0.5, 0.25, 0.125)

julia> function sum_cosines(x, y, z)
         return cos(x) + cos(y) + cos(z)
       end;

julia> @btime sum_cosines($a, $b, $c);  # the `;` suppresses printing the returned value
  11.899 ns (0 allocations: 0 bytes)    # calling the function takes ~12 ns (nanoseconds)
                                        # the function does not allocate any memory

# if we omit the '$', what we see is misleading
julia> @btime sum_cosines(a, b, c);    # the function appears more than twice slower 
 28.441 ns (1 allocation: 16 bytes)    # the function appears to be allocating memory

# @benchmark can be used the same way that @btime is used
julia> @benchmark sum_cosines($a,$b,$c) # do not use a ';' here
BenchmarkTools.Trial:
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     12.111 ns (0.00% GC)
  median time:      12.213 ns (0.00% GC)
  mean time:        12.500 ns (0.00% GC)
  maximum time:     39.741 ns (0.00% GC)
  --------------
  samples:          1500
  evals/sample:     999

虽然有一些参数可以调整，但默认值通常效果很好.有关面向经验丰富的用户的 BenchmarkTools 的更多信息，请参阅手册.

While there are parameters than can be adjusted, the default values usually work well. For additional information about BenchmarkTools for experienced ursers, see the manual.

这篇关于我可以通过哪些方式对 Julia 函数进行基准测试?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！