UNIX“时间"命令对于基准测试是否足够准确? [英] Is the UNIX `time` command accurate enough for benchmarks?

问题描述

假设我想对两个程序进行基准测试:foo.py和bar.py.

Let's say I wanted to benchmark two programs: foo.py and bar.py.

是否进行了数千次运行，并且time python foo.py和time python bar.py各自的平均值足以分析和比较它们的速度?


另外，如果每个程序的执行时间不到一秒(假设不是以上所述)，time仍然可以使用吗?

Are a couple thousand runs and the respective averages of time python foo.py and time python bar.py adequate enough for profiling and comparing their speed?


Additionally, if the execution of each program was sub-second (assume it wasn't for the above), would time still be okay to use?

推荐答案

time对于运行一秒钟的基准测试产生了足够好的时间，否则exec()处理某个流程所花费的时间与其运行时间相比可能会很大.时间.

time produces good enough times for benchmarks that run over one second otherwise the time it took exec()ing a process may be large compared to its run-time.

但是，在进行基准测试时，应该注意上下文切换.也就是说，另一个进程可能正在使用CPU，因此与基准测试竞争CPU并增加了其运行时间.为了避免与其他进程争用，您应该运行这样的基准测试:

However, when benchmarking you should watch out for context switching. That is, another process may be using CPU thus contending for CPU with your benchmark and increasing its run time. To avoid contention with other processes you should run a benchmark like this:

sudo chrt -f 99 /usr/bin/time --verbose <benchmark>

或

sudo chrt -f 99 perf stat -ddd <benchmark>

sudo chrt -f 99在FIFO实时类中以优先级99运行基准，这使您的进程成为最高优先级的进程，并避免了上下文切换(您可以更改/etc/security/limits.conf，因此它不需要特权进程即可)使用实时优先级).

sudo chrt -f 99 runs your benchmark in FIFO real-time class with priority 99, which makes your process the top priority process and avoids context switching (you can change your /etc/security/limits.conf so that it doesn't require a privileged process to use real-time priorities).

这还会使time报告所有可用的统计信息，包括基准测试发生的上下文切换次数，通常应为0，否则您可能希望重新运行基准测试.

It also makes time report all the available stats, including the number of context switches your benchmark incurred, which should normally be 0, otherwise you may like to rerun the benchmark.

perf stat -ddd比/usr/bin/time更具信息性，并显示诸如每周期指令，分支和高速缓存未命中等信息.

perf stat -ddd is even more informative than /usr/bin/time and displays such information as instructions-per-cycle, branch and cache misses, etc.

最好禁用CPU频率缩放和增强功能，以使CPU频率在基准测试期间保持恒定，以获得一致的结果.

And it is better to disable the CPU frequency scaling and boost, so that the CPU frequency stays constant during the benchmark to get consistent results.

这篇关于UNIX“时间"命令对于基准测试是否足够准确?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！