UNIX `time` 命令对于基准测试是否足够准确? [英] Is the UNIX `time` command accurate enough for benchmarks?

问题描述

假设我想对两个程序进行基准测试:foo.py 和 bar.py.

Let's say I wanted to benchmark two programs: foo.py and bar.py.

几千次运行以及 time python foo.py 和 time python bar.py 的各自平均值是否足以分析和比较它们的速度?


另外，如果每个程序的执行都是亚秒级的(假设不是上面的)，time还能用吗?

Are a couple thousand runs and the respective averages of time python foo.py and time python bar.py adequate enough for profiling and comparing their speed?


Additionally, if the execution of each program was sub-second (assume it wasn't for the above), would time still be okay to use?

推荐答案

time 为运行超过一秒的基准测试产生足够好的时间，否则花费的时间 exec() 一个进程与其运行时相比可能很大.

time produces good enough times for benchmarks that run over one second otherwise the time it took exec()ing a process may be large compared to its run-time.

但是，在进行基准测试时，您应该注意上下文切换.也就是说，另一个进程可能正在使用 CPU，从而与您的基准测试竞争 CPU 并增加其运行时间.为避免与其他进程发生争用，您应该运行如下基准:

However, when benchmarking you should watch out for context switching. That is, another process may be using CPU thus contending for CPU with your benchmark and increasing its run time. To avoid contention with other processes you should run a benchmark like this:

sudo chrt -f 99 /usr/bin/time --verbose <benchmark>

或

sudo chrt -f 99 perf stat -ddd <benchmark>

sudo chrt -f 99 在优先级为 99 的 FIFO 实时类中运行您的基准测试，这使您的进程成为最高优先级进程并避免上下文切换(您可以更改您的 /etc/security/limits.conf 以便它不需要特权进程来使用实时优先级).

sudo chrt -f 99 runs your benchmark in FIFO real-time class with priority 99, which makes your process the top priority process and avoids context switching (you can change your /etc/security/limits.conf so that it doesn't require a privileged process to use real-time priorities).

它还使 time 报告所有可用的统计信息，包括您的基准测试发生的上下文切换次数，通常应为 0，否则您可能想重新运行基准测试.

It also makes time report all the available stats, including the number of context switches your benchmark incurred, which should normally be 0, otherwise you may like to rerun the benchmark.

perf stat -ddd 比 /usr/bin/time 提供更多信息，并显示诸如每周期指令、分支和缓存未命中等信息.

perf stat -ddd is even more informative than /usr/bin/time and displays such information as instructions-per-cycle, branch and cache misses, etc.

并且最好禁用 CPU 频率缩放和提升，以便在基准测试期间 CPU 频率保持恒定以获得一致的结果.

And it is better to disable the CPU frequency scaling and boost, so that the CPU frequency stays constant during the benchmark to get consistent results.

这篇关于UNIX `time` 命令对于基准测试是否足够准确?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！