我可以通过哪种方式对Julia函数进行基准测试? [英] In what way(s) can I benchmark a Julia function?

问题描述

我已经自学了机器学习，最近开始研究Julia机器学习生态系统.

---

来自python背景，并具有一些Tensorflow和 OpenCV / skimage 经验，我想对Julia ML库进行基准测试 (Flux/JuliaImages)相对于同行，以了解其执行 CV (任何)任务的速度有多快，并决定我是否应该转向使用Julia.

I've self-taught myself machine learning and have recently started delving into the Julia Machine Learning Ecosystem.

---

Coming from a python background and having some Tensorflow and OpenCV/skimage experience, I want to benchmark Julia ML libraries (Flux/JuliaImages) against its counterparts to see how fast or slow it really performs CV(any) task(s) and to decide if I should shift to using Julia.

我知道如何使用 timeit来获取在python中执行功能所花费的时间. 模块是这样的:

I know how to get the time taken to execute a function in python using timeit module like this :

#Loading an Image using OpenCV

s = """\
img = cv2.imread('sample_image.png', 1)
"""
setup = """\
import timeit
"""
print(str(round((timeit.timeit(stmt = s, setup = setup, number = 1))*1000, 2)) + " ms")
#printing the time taken in ms rounded to 2 digits

如何使用适当的库(在本例中为JuliaImages)比较在Julia中执行相同任务的功能的执行时间.

How does one compare the execution time of a function performing the same task in Julia using the appropriate library (in this case, JuliaImages).

Julia是否提供时间/基准的任何功能/宏?

Does Julia provide any function/macro to time/benchmark ?

推荐答案

using BenchmarkTools是对Julia函数进行基准测试的推荐方法.除非您要花一些时间来安排时间，否则请使用@benchmark或从其导出的不太冗长的@btime宏.由于这些宏背后的机制会多次评估目标函数，因此@time对于基准测试运行缓慢的事物很有用(例如，涉及磁盘访问或非常耗时的计算).

using BenchmarkTools is the recommended way to benchmark Julia functions. Unless you are timing something that takes quite a while, use either @benchmark or the less verbose @btime macros exported from it. Because the machinery behind these macros evaluates the target function many times, @time is useful for benchmarking things that run slowly (e.g. where disk access or very time-consuming calculations are involved).

正确使用@btime或@benchmark很重要，这样可以避免产生误导性的结果.通常，您正在对采用一个或多个参数的函数进行基准测试.进行基准测试时，所有参数都应为外部变量:(无基准测试宏)

It is important to use @btime or @benchmark correctly, this avoids misleading results. Usually, you are benchmarking a function that takes one or more arguments. When benchmarking, all arguments should be external variables: (without the benchmark macro)

x = 1
f(x)
# do not use f(1)

该功能将被评估多次.为了防止在每次评估函数时重新评估函数自变量，我们必须通过在每个用作自变量的变量的名称前加上$来标记每个自变量.基准测试宏使用此方法指示在基准测试过程开始时应对变量进行一次评估(解析)，然后按原样直接重用结果:

The function will be evaluated many times. To prevent the function arguments from being re-evaluated whenever the function is evaluated, we must mark each argument by prefixing a $ to the name of each variable that is used as an argument. The benchmarking macros use this to indicate that the variable should be evaluated (resolved) once, at the start of the benchmarking process and then the result is to be reused directly as is:

julia> using BenchmarkTools
julia> a = 1/2;
julia> b = 1/4;
julia> c = 1/8;
julia> a, b, c
(0.5, 0.25, 0.125)

julia> function sum_cosines(x, y, z)
         return cos(x) + cos(y) + cos(z)
       end;

julia> @btime sum_cosines($a, $b, $c);  # the `;` suppresses printing the returned value
  11.899 ns (0 allocations: 0 bytes)    # calling the function takes ~12 ns (nanoseconds)
                                        # the function does not allocate any memory

# if we omit the '$', what we see is misleading
julia> @btime sum_cosines(a, b, c);    # the function appears more than twice slower 
 28.441 ns (1 allocation: 16 bytes)    # the function appears to be allocating memory

# @benchmark can be used the same way that @btime is used
julia> @benchmark sum_cosines($a,$b,$c) # do not use a ';' here
BenchmarkTools.Trial:
  memory estimate:  0 bytes
  allocs estimate:  0
  --------------
  minimum time:     12.111 ns (0.00% GC)
  median time:      12.213 ns (0.00% GC)
  mean time:        12.500 ns (0.00% GC)
  maximum time:     39.741 ns (0.00% GC)
  --------------
  samples:          1500
  evals/sample:     999

虽然有一些参数可以调整，但是默认值通常效果很好.有关面向经验丰富的使用者的BenchmarkTools的其他信息，请参见手册.

While there are parameters than can be adjusted, the default values usually work well. For additional information about BenchmarkTools for experienced ursers, see the manual.

这篇关于我可以通过哪种方式对Julia函数进行基准测试?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！