针对短代码获得可靠的性能测量 [英] Getting reliable performance measurements for short bits of code

问题描述

我正在尝试剖析一些函数，这些函数以不同的方式实现同​​一算法的不同版本.我增加了每个函数的运行次数，以使单个函数所花费的总时间大约为1分钟(以揭示性能差异).

I'm trying to profile a few set of functions which implement different versions of the same algorithm in different ways. I've increased the number of times each function is run so that the total time spent in a single function is roughly 1 minute (to reveal performance differences).

现在，运行几次测试会产生令人困惑的结果.同一功能的多个执行之间存在很大的差异(±50％)，因此几乎不可能确定哪个功能最快(这是测试的目标).

Now, running several times the test produces baffling results. There is a huge variability (+- 50 %) between several executions of the same function, and determining which function is the fastest (which is the goal of the test) is nearly impossible because of that.

在运行测试之前，我是否应该特别注意一些事项，以便获得更平滑的测量结果?如果失败，是否要多次运行测试并计算每种功能的平均值?

Is there something special I should take care of before running the tests, so that I get smoother measurements? Failing that, is running the test several times and compute the average for each function the way to go?

推荐答案

要检查的东西很多！

首先，请确保您的功能实际上受CPU限制.如果是这样，请确保已禁用所有CPU节流，turbo模式和省电模式(在BIOS中)以进行测试.如果仍然有问题，请尝试将您的进程固定在单个内核上.也许也禁用超线程.

First, make sure your functions are actually CPU-bound. If so, make sure you have all CPU throttling, turbo modes, and power-saving modes disabled (in BIOS) for the test. If you still have trouble, try pinning your process to a single core. Disable hyper-threading too perhaps.

所有这一切的目的是确保您的代码在单个内核上热运行，而不会造成太多干扰.如果您使用的是Linux，则可以从可用内核的OS列表中删除一个内核并使用它(这样就不会对该内核造成干扰).

The goal of all this is to make sure you get your code running hot on a single core without much interruption. If you're on Linux, you can remove a single core from the OS list of available cores and use that (so there is no chance of interference on that core).

多次运行测试是一个好主意，但使用平均值(算术平均值)则不是.取而代之的是，使用中值或最小值或其他不受异常值影响的度量.通常，偶尔的长时间测试运行可能会被完全淘汰(除非您正在构建实时系统！).

Running the test several times is a good idea, but using the average (arithmetic mean) is not. Instead, use the median or minimum or some other measurement which won't be influenced by outliers. Usually, the occasional long test run can be thrown out entirely (unless you're building a real-time system!).

这篇关于针对短代码获得可靠的性能测量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！