为什么比未排序的数组更快地处理排序的数组？ [英] Why is it faster to process a sorted array than an unsorted array?
问题描述
这里是一段看起来非常奇特的C ++代码。由于一些奇怪的原因，奇迹地排序数据使得代码的速度快了六倍。
#include < algorithm>
#include< ctime>
#include< iostream>
int main（）
{
//生成数据
const unsigned arraySize = 32768;
int data [arraySize];
for（unsigned c = 0; c
。
如上所示，罪魁祸首是if语句：
if（data [c]> = 128）
sum + = data [c] ;
请注意，数据均匀分布在0和255之间。
当数据排序，大致上半个迭代将不会进入if语句。之后，他们都将输入if语句。
这是非常友好的分支预测器，因为分支连续多次往同一个方向多次。
快速可视化： / p>
T =分支
N =不分支
data [] = 0， ...
branch = NNNNN ... NNTTT ... TTT ... 。
= NNNNNNNNNNNN ... NNNNNNNTTTTTTTTT ... TTTTTTTTTT（容易预测）
然而，当数据完全随机时，分支预测器被赋予无用性，因为它不能预测随机数据。
因此，可能会有大约50％的误预测。 （不比随机猜测更好）
data [] = 226,185,125,158,198,144,217,79 ，202,118,14,150,177,182,133，...
branch = T，T，N，T，T，T，T，N，T，N，N，T，T， T，N ...
= TTNTTTTNTNNTTTN ...（完全随机 - 难以预测）
替换：
$ b b
if（data [c]> = 128）
sum + = data [c]
其中：
code> int t =（data [c] -128）>> 31;
sum + =〜t&数据[c];
这消除了分支，并用一些按位操作替换它。
（注意，这个hack并不完全等同于原来的if语句，但在这种情况下，它对于 data [] 。）
基准：Core i7 920 @ 3.5 GHz
C ++ - Visual Studio 2010 - x64版本
//分支 - 随机
秒= 11.777
//分支 - 已排序
秒= 2.352
//无分支 - 随机
秒= 2.564
//无分支 - 排序
秒= 2.587
Java - Netbeans 7.1.1 JDK 7 - x64
//分支 - 随机
秒= 10.93293813
//分支 - 已排序
seconds = 5.643797077
//无分支 - 随机
秒= 3.113581453
//无分支 - 已排序
秒= 3.186068823
观察：
- 使用分支：已排序和未排序的数据之间存在巨大差异。
- 使用Hack：数据。
- 在C ++的情况下，当数据排序时，hack实际上比分支慢一些。
一般的经验法则是避免关键循环中的数据相关分支。 （例如在此示例中）
更新：
-
GCC 4.6.1 with
-O3或-ftree-vectorizeon x64能够生成条件移动。 -
即使在<$ c下，VC ++ 2010也无法为此分支生成条件移动$ c> / Ox 。
-
Intel Compiler 11做了奇迹。它互换两个循环，从而将不可预测的分支提升到外部循环。因此，它不仅免疫错误预测，它也是任何VC ++和GCC可以生成的两倍快！换句话说，ICC利用测试循环来击败基准测试...
-
如果你给英特尔编译器无分支代码，
这表明，即使成熟的现代编译器也可以在优化代码的能力上有很大的不同...
Here is a piece of C++ code that seems very peculiar. For some strange reason, sorting the data miraculously makes the code almost six times faster.
#include <algorithm>
#include <ctime>
#include <iostream>
int main()
{
// Generate data
const unsigned arraySize = 32768;
int data[arraySize];
for (unsigned c = 0; c < arraySize; ++c)
data[c] = std::rand() % 256;
// !!! With this, the next loop runs faster
std::sort(data, data + arraySize);
// Test
clock_t start = clock();
long long sum = 0;
for (unsigned i = 0; i < 100000; ++i)
{
// Primary loop
for (unsigned c = 0; c < arraySize; ++c)
{
if (data[c] >= 128)
sum += data[c];
}
}
double elapsedTime = static_cast<double>(clock() - start) / CLOCKS_PER_SEC;
std::cout << elapsedTime << std::endl;
std::cout << "sum = " << sum << std::endl;
}
- Without
std::sort(data, data + arraySize);, the code runs in 11.54 seconds. - With the sorted data, the code runs in 1.93 seconds.
Initially, I thought this might be just a language or compiler anomaly. So I tried it in Java.
import java.util.Arrays;
import java.util.Random;
public class Main
{
public static void main(String[] args)
{
// Generate data
int arraySize = 32768;
int data[] = new int[arraySize];
Random rnd = new Random(0);
for (int c = 0; c < arraySize; ++c)
data[c] = rnd.nextInt() % 256;
// !!! With this, the next loop runs faster
Arrays.sort(data);
// Test
long start = System.nanoTime();
long sum = 0;
for (int i = 0; i < 100000; ++i)
{
// Primary loop
for (int c = 0; c < arraySize; ++c)
{
if (data[c] >= 128)
sum += data[c];
}
}
System.out.println((System.nanoTime() - start) / 1000000000.0);
System.out.println("sum = " + sum);
}
}
With a somewhat similar but less extreme result.
My first thought was that sorting brings the data into the cache, but then I thought how silly that is because the array was just generated.
- What is going on?
- Why is a sorted array faster to process than an unsorted array?
- The code is summing up some independent terms, and the order should not matter.
You are a victim of branch prediction fail.
What is Branch Prediction?
Consider a railroad junction:
Image by Mecanismo, via Wikimedia Commons. Used under the CC-By-SA 3.0 license.
Now for the sake of argument, suppose this is back in the 1800s - before long distance or radio communication.
You are the operator of a junction and you hear a train coming. You have no idea which way it is supposed to go. You stop the train to ask the driver which direction they want. And then you set the switch appropriately.
Trains are heavy and have a lot of inertia. So they take forever to start up and slow down.
Is there a better way? You guess which direction the train will go!
- If you guessed right, it continues on.
- If you guessed wrong, the captain will stop, back up, and yell at you to flip the switch. Then it can restart down the other path.
If you guess right every time, the train will never have to stop.
If you guess wrong too often, the train will spend a lot of time stopping, backing up, and restarting.
Consider an if-statement: At the processor level, it is a branch instruction:
You are a processor and you see a branch. You have no idea which way it will go. What do you do? You halt execution and wait until the previous instructions are complete. Then you continue down the correct path.
Modern processors are complicated and have long pipelines. So they take forever to "warm up" and "slow down".
Is there a better way? You guess which direction the branch will go!
- If you guessed right, you continue executing.
- If you guessed wrong, you need to flush the pipeline and roll back to the branch. Then you can restart down the other path.
If you guess right every time, the execution will never have to stop.
If you guess wrong too often, you spend a lot of time stalling, rolling back, and restarting.
This is branch prediction. I admit it's not the best analogy since the train could just signal the direction with a flag. But in computers, the processor doesn't know which direction a branch will go until the last moment.
So how would you strategically guess to minimize the number of times that the train must back up and go down the other path? You look at the past history! If the train goes left 99% of the time, then you guess left. If it alternates, then you alternate your guesses. If it goes one way every 3 times, you guess the same...
In other words, you try to identify a pattern and follow it. This is more or less how branch predictors work.
Most applications have well-behaved branches. So modern branch predictors will typically achieve >90% hit rates. But when faced with unpredictable branches with no recognizable patterns, branch predictors are virtually useless.
Further reading: "Branch predictor" article on Wikipedia.
As hinted from above, the culprit is this if-statement:
if (data[c] >= 128)
sum += data[c];
Notice that the data is evenly distributed between 0 and 255. When the data is sorted, roughly the first half of the iterations will not enter the if-statement. After that, they will all enter the if-statement.
This is very friendly to the branch predictor since the branch consecutively goes the same direction many times. Even a simple saturating counter will correctly predict the branch except for the few iterations after it switches direction.
Quick visualization:
T = branch taken
N = branch not taken
data[] = 0, 1, 2, 3, 4, ... 126, 127, 128, 129, 130, ... 250, 251, 252, ...
branch = N N N N N ... N N T T T ... T T T ...
= NNNNNNNNNNNN ... NNNNNNNTTTTTTTTT ... TTTTTTTTTT (easy to predict)
However, when the data is completely random, the branch predictor is rendered useless because it can't predict random data. Thus there will probably be around 50% misprediction. (no better than random guessing)
data[] = 226, 185, 125, 158, 198, 144, 217, 79, 202, 118, 14, 150, 177, 182, 133, ...
branch = T, T, N, T, T, T, T, N, T, N, N, T, T, T, N ...
= TTNTTTTNTNNTTTN ... (completely random - hard to predict)
So what can be done?
If the compiler isn't able to optimize the branch into a conditional move, you can try some hacks if you are willing to sacrifice readability for performance.
Replace:
if (data[c] >= 128)
sum += data[c];
with:
int t = (data[c] - 128) >> 31;
sum += ~t & data[c];
This eliminates the branch and replaces it with some bitwise operations.
(Note that this hack is not strictly equivalent to the original if-statement. But in this case, it's valid for all the input values of data[].)
Benchmarks: Core i7 920 @ 3.5 GHz
C++ - Visual Studio 2010 - x64 Release
// Branch - Random
seconds = 11.777
// Branch - Sorted
seconds = 2.352
// Branchless - Random
seconds = 2.564
// Branchless - Sorted
seconds = 2.587
Java - Netbeans 7.1.1 JDK 7 - x64
// Branch - Random
seconds = 10.93293813
// Branch - Sorted
seconds = 5.643797077
// Branchless - Random
seconds = 3.113581453
// Branchless - Sorted
seconds = 3.186068823
Observations:
- With the Branch: There is a huge difference between the sorted and unsorted data.
- With the Hack: There is no difference between sorted and unsorted data.
- In the C++ case, the hack is actually a tad slower than with the branch when the data is sorted.
A general rule of thumb is to avoid data-dependent branching in critical loops. (such as in this example)
Update :
GCC 4.6.1 with
-O3or-ftree-vectorizeon x64 is able to generate a conditional move. So there is no difference between the sorted and unsorted data - both are fast.VC++ 2010 is unable to generate conditional moves for this branch even under
/Ox.Intel Compiler 11 does something miraculous. It interchanges the two loops, thereby hoisting the unpredictable branch to the outer loop. So not only is it immune the mispredictions, it is also twice as fast as whatever VC++ and GCC can generate! In other words, ICC took advantage of the test-loop to defeat the benchmark...
If you give the Intel Compiler the branchless code, it just out-right vectorizes it... and is just as fast as with the branch (with the loop interchange).
This goes to show that even mature modern compilers can vary wildly in their ability to optimize code...
这篇关于为什么比未排序的数组更快地处理排序的数组？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
