什么是微基准测试？ [英] What is microbenchmarking?

问题描述

我听说过这个术语，但我不完全确定它是什么意思，所以：

I've heard this term used, but I'm not entirely sure what it means, so:

• 它是什么意思什么不是什么意思？


• 什么是IS和IS不是微基准测试的例子？


• 微基准测试的危险是什么？你怎么避免它？
  
  
  
  
  • （或者这是件好事吗？）
  
  

  推荐答案

  这正是它在锡罐上所说的 - 它测量的是小的性能，就像系统调用内核一样操作系统。

  It means exactly what it says on the tin can - it's measuring the performance of something "small", like a system call to the kernel of an operating system.

  危险在于人们可能会使用从微基准测试中获得的任何结果来指示优化。众所周知：

  The danger is that people may use whatever results they obtain from microbenchmarking to dictate optimizations. And as we all know:

  
  

  我们应该忘记效率低，大约97％的时间说：过早优化是$的根源b $ b所有邪恶 - Donald Knuth

  We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil" -- Donald Knuth

  可能有许多因素扭曲了微基准测试的结果。编译器优化是其中之一如果被测量的操作花费的时间太少，以至于无论你用什么测量它都需要比实际操作本身更长的时间，你的微基准测试也会有所偏差。

  There can be many factors that skew the result of microbenchmarks. Compiler optimizations is one of them. If the operation being measured takes so little time that whatever you use to measure it takes longer than the actual operation itself, your microbenchmarks will be skewed also.

  例如，有人可能会采用开销的微基准来获取循环：

  For example, someone might take a microbenchmark of the overhead of for loops:

  void TestForLoop()
  {
      time start = GetTime();
  
      for(int i = 0; i < 1000000000; ++i)
      {
      }
  
      time elapsed = GetTime() - start;
      time elapsedPerIteration = elapsed / 1000000000;
      printf("Time elapsed for each iteration: %d\n", elapsedPerIteration);
  }
  

  显然，编译器可以看到循环完全没有任何东西，并且根本不会为循环生成任何代码。所以已过去的价值和 elapsedPerIteration 几乎没用。

  Obviously compilers can see that the loop does absolutely nothing and not generate any code for the loop at all. So the value of elapsed and elapsedPerIteration is pretty much useless.

  即使循环执行某些操作：

  Even if the loop does something:

  void TestForLoop()
  {
      int sum = 0;
      time start = GetTime();
  
      for(int i = 0; i < 1000000000; ++i)
      {
          ++sum;
      }
  
      time elapsed = GetTime() - start;
      time elapsedPerIteration = elapsed / 1000000000;
      printf("Time elapsed for each iteration: %d\n", elapsedPerIteration);
  }
  

  编译器可能会看到变量 sum 不会用于任何事情并将其优化掉，并优化掉for循环。可是等等！如果我们这样做会怎样：

  The compiler may see that the variable sum isn't going to be used for anything and optimize it away, and optimize away the for loop as well. But wait! What if we do this:

  void TestForLoop()
  {
      int sum = 0;
      time start = GetTime();
  
      for(int i = 0; i < 1000000000; ++i)
      {
          ++sum;
      }
  
      time elapsed = GetTime() - start;
      time elapsedPerIteration = elapsed / 1000000000;
      printf("Time elapsed for each iteration: %d\n", elapsedPerIteration);
      printf("Sum: %d\n", sum); // Added
  }
  

  编译器可能足够聪明，可以实现 sum 将始终是一个常数值，并优化所有这些。很多人会对编译器的优化功能感到惊讶。

  The compiler might be smart enough to realize that sum will always be a constant value, and optimize all that away as well. Many would be surprised at the optimizing capabilities of compilers these days.

  但编译器无法优化的东西呢？

  But what about things that compilers can't optimize away?

  void TestFileOpenPerformance()
  {
      FILE* file = NULL;
      time start = GetTime();
  
      for(int i = 0; i < 1000000000; ++i)
      {
          file = fopen("testfile.dat");
          fclose(file);
      }
  
      time elapsed = GetTime() - start;
      time elapsedPerIteration = elapsed / 1000000000;
      printf("Time elapsed for each file open: %d\n", elapsedPerIteration);
  }
  

  即使这不是一个有用的测试！操作系统可能会看到文件被非常频繁地打开，因此它可能会将其预加载到内存中以提高性能。几乎所有操作系统都这样做。打开应用程序时会发生同样的事情 - 操作系统可能会找出打开最多的前5个应用程序，并在启动计算机时将应用程序代码预加载到内存中！

  Even this is not a useful test! The operating system may see that the file is being opened very frequently, so it may preload it in memory to improve performance. Pretty much all operating systems do this. The same thing happens when you open applications - operating systems may figure out the top ~5 applications you open the most and preload the application code in memory when you boot up the computer!

  事实上，有无数变量发挥作用：引用的局部性（例如数组与链表），缓存和内存带宽的影响，编译器内联，编译器实现，编译器开关，处理器内核数量，优化处理器级别，操作系统调度程序，操作系统后台进程等。

  In fact, there are countless variables that come into play: locality of reference (e.g. arrays vs. linked lists), effects of caches and memory bandwidth, compiler inlining, compiler implementation, compiler switches, number of processor cores, optimizations at the processor level, operating system schedulers, operating system background processes, etc.

  因此，在许多情况下，微基准测试并不是一个非常有用的指标。它绝对不能用明确定义的测试用例（分析）取代整个程序基准。首先编写可读代码，然后编写配置文件以查看需要执行的操作（如果有）。

  So microbenchmarking isn't exactly a useful metric in a lot of cases. It definitely does not replace whole-program benchmarks with well-defined test cases (profiling). Write readable code first, then profile to see what needs to be done, if any.

  我想强调微基准测试本身并不是邪恶本身，但是必须仔细使用它们（对于许多与计算机相关的其他事情都是如此）

  I would like to emphasize that microbenchmarks are not evil per se, but one has to use them carefully (that's true for lots of other things related to computers)

  这篇关于什么是微基准测试？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！