如何在 Java 中编写正确的微基准测试? [英] How do I write a correct micro-benchmark in Java?
问题描述
你如何用 Java 编写(和运行)正确的微基准测试?
How do you write (and run) a correct micro-benchmark in Java?
我正在寻找一些代码示例和注释来说明需要考虑的各种事情.
I'm looking for some code samples and comments illustrating various things to think about.
示例:基准测试应该测量时间/迭代还是迭代/时间，为什么?
Example: Should the benchmark measure time/iteration or iterations/time, and why?
推荐答案
关于编写微基准测试的提示 来自 Java HotSpot 的创建者:
Tips about writing micro benchmarks from the creators of Java HotSpot:
规则 0:阅读有关 JVM 和微基准测试的知名论文.一个很好的例子是 Brian Goetz，2005 年.不要对微基准期望过高；它们仅测量有限范围的 JVM 性能特征.
Rule 0: Read a reputable paper on JVMs and micro-benchmarking. A good one is Brian Goetz, 2005. Do not expect too much from micro-benchmarks; they measure only a limited range of JVM performance characteristics.
规则 1: 始终包括一个预热阶段，该阶段一直运行您的测试内核，足以在计时阶段之前触发所有初始化和编译.(预热阶段的迭代次数较少.经验法则是数万次内循环迭代.)
Rule 1: Always include a warmup phase which runs your test kernel all the way through, enough to trigger all initializations and compilations before timing phase(s). (Fewer iterations is OK on the warmup phase. The rule of thumb is several tens of thousands of inner loop iterations.)
规则 2: 始终使用 -XX:+PrintCompilation、-verbose:gc 等运行，以便您可以验证编译器和 JVM 的其他部分在您的计时阶段没有做意外的工作.
Rule 2: Always run with -XX:+PrintCompilation, -verbose:gc, etc., so you can verify that the compiler and other parts of the JVM are not doing unexpected work during your timing phase.
规则 2.1: 在计时和预热阶段的开始和结束时打印消息，以便您可以验证在计时阶段没有规则 2 的输出.
Rule 2.1: Print messages at the beginning and end of timing and warmup phases, so you can verify that there is no output from Rule 2 during the timing phase.
规则 3: 注意 -client 和 -server 之间的区别，以及 OSR 和常规编译之间的区别.-XX:+PrintCompilation 标志报告 OSR 编译时使用 at 符号表示非初始入口点，例如:Trouble$1::run@2(41 字节)代码>.如果您追求最佳性能，则更喜欢服务器而不是客户端，常规而不是 OSR.
Rule 3: Be aware of the difference between -client and -server, and OSR and regular compilations. The -XX:+PrintCompilation flag reports OSR compilations with an at-sign to denote the non-initial entry point, for example: Trouble$1::run @ 2 (41 bytes). Prefer server to client, and regular to OSR, if you are after best performance.
规则 4: 注意初始化效果.不要在计时阶段第一次打印，因为打印会加载和初始化类.不要在预热阶段(或最终报告阶段)之外加载新类，除非您专门测试类加载(在这种情况下只加载测试类).规则 2 是您抵御此类影响的第一道防线.
Rule 4: Be aware of initialization effects. Do not print for the first time during your timing phase, since printing loads and initializes classes. Do not load new classes outside of the warmup phase (or final reporting phase), unless you are testing class loading specifically (and in that case load only the test classes). Rule 2 is your first line of defense against such effects.
规则 5:注意反优化和重新编译的影响.在计时阶段不要第一次使用任何代码路径，因为编译器可能会根据早先的乐观假设，即根本不会使用该路径，而重新编译代码.规则 2 是您抵御此类影响的第一道防线.
Rule 5: Be aware of deoptimization and recompilation effects. Do not take any code path for the first time in the timing phase, because the compiler may junk and recompile the code, based on an earlier optimistic assumption that the path was not going to be used at all. Rule 2 is your first line of defense against such effects.
规则 6: 使用适当的工具来读取编译器的思想，并期望对它生成的代码感到惊讶.在形成关于什么使某事更快或更慢的理论之前，自己检查代码.
Rule 6: Use appropriate tools to read the compiler's mind, and expect to be surprised by the code it produces. Inspect the code yourself before forming theories about what makes something faster or slower.
规则 7: 减少测量中的噪声.在安静的机器上运行您的基准测试，并运行几次，丢弃异常值.使用-Xbatch 将编译器与应用程序序列化，并考虑设置-XX:CICompilerCount=1 以防止编译器与自身并行运行.尽量减少GC开销，设置Xmx(足够大)等于Xms并使用UseEpsilonGC 如果可用.
Rule 7: Reduce noise in your measurements. Run your benchmark on a quiet machine, and run it several times, discarding outliers. Use -Xbatch to serialize the compiler with the application, and consider setting -XX:CICompilerCount=1 to prevent the compiler from running in parallel with itself. Try your best to reduce GC overhead, set Xmx(large enough) equals Xms and use UseEpsilonGC if it is available.
规则 8: 将库用于您的基准测试，因为它可能更高效，并且已经针对此目的进行了调试.如JMH、Caliper 或 Bill 和Paul 的优秀 UCSD Java 基准测试.
Rule 8: Use a library for your benchmark as it is probably more efficient and was already debugged for this sole purpose. Such as JMH, Caliper or Bill and Paul's Excellent UCSD Benchmarks for Java.
这篇关于如何在 Java 中编写正确的微基准测试?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
