如何在Java中编写正确的微基准测试？ [英] How do I write a correct micro-benchmark in Java?

问题描述

你如何在Java中编写（并运行）正确的微基准？

How do you write (and run) a correct micro-benchmark in Java?

我在这里寻找代码示例和注释，说明要考虑的各种事项。

I'm looking here for code samples and comments illustrating various things to think about.

示例：基准测量应该测量时间/迭代或迭代/时间，以及为什么？

Example: Should the benchmark measure time/iteration or iterations/time, and why?

相关： 秒表基准测试是否可以接受？

推荐答案

关于从Java HotSpot创建者编写微基准的提示：

Tips about writing micro benchmarks from the creators of Java HotSpot:

规则0：阅读有关JVM和微基准测试的着名论文。一个好的方法是 Brian Goetz，2005 。微观基准不要期望太多;它们仅测量有限范围的JVM性能特征。

Rule 0: Read a reputable paper on JVMs and micro-benchmarking. A good one is Brian Goetz, 2005. Do not expect too much from micro-benchmarks; they measure only a limited range of JVM performance characteristics.

规则1：始终包含一个运行测试内核的预热阶段，足以在计时阶段之前触发所有初始化和编译。 （在预热阶段，迭代次数较少。经验法则是数万次内循环迭代。）

Rule 1: Always include a warmup phase which runs your test kernel all the way through, enough to trigger all initializations and compilations before timing phase(s). (Fewer iterations is OK on the warmup phase. The rule of thumb is several tens of thousands of inner loop iterations.)

规则2：始终使用 -XX：+ PrintCompilation ， -verbose：gc 等运行，以便您可以验证编译器JVM的其他部分在您的计时阶段没有意外工作。

Rule 2: Always run with -XX:+PrintCompilation, -verbose:gc, etc., so you can verify that the compiler and other parts of the JVM are not doing unexpected work during your timing phase.

规则2.1：在计时的开始和结束时打印消息和预热阶段，因此您可以在计时阶段验证规则2中没有输出。

Rule 2.1: Print messages at the beginning and end of timing and warmup phases, so you can verify that there is no output from Rule 2 during the timing phase.

规则3：请注意-client和-server，OSR和常规编译之间的区别。 -XX：+ PrintCompilation 标志报告带有at符号的OSR编译，表示非初始入口点，例如： Trouble $ 1 ::运行@ 2（41字节）。如果您追求最佳性能，则首选服务器到客户端，并定期到OSR。

Rule 3: Be aware of the difference between -client and -server, and OSR and regular compilations. The -XX:+PrintCompilation flag reports OSR compilations with an at-sign to denote the non-initial entry point, for example: Trouble$1::run @ 2 (41 bytes). Prefer server to client, and regular to OSR, if you are after best performance.

规则4：请注意初始化效果。在打印加载和初始化类时，不要在计时阶段第一次打印。除非您专门测试类加载（并且在这种情况下仅加载测试类），否则不要在预热阶段（或最终报告阶段）之外加载新类。规则2是您抵御此类影响的第一道防线。

Rule 4: Be aware of initialization effects. Do not print for the first time during your timing phase, since printing loads and initializes classes. Do not load new classes outside of the warmup phase (or final reporting phase), unless you are testing class loading specifically (and in that case load only the test classes). Rule 2 is your first line of defense against such effects.

规则5：了解去优化和重新编译效果。在计时阶段第一次不要采用任何代码路径，因为编译器可能会破坏并重新编译代码，这是基于先前的乐观假设，即路径根本不会被使用。规则2是您抵御此类影响的第一道防线。

Rule 5: Be aware of deoptimization and recompilation effects. Do not take any code path for the first time in the timing phase, because the compiler may junk and recompile the code, based on an earlier optimistic assumption that the path was not going to be used at all. Rule 2 is your first line of defense against such effects.

规则6：使用适当的工具来阅读编译器的想法，并期望成为对它产生的代码感到惊讶。在形成关于什么使得更快或更慢的东西的理论之前，自己检查代码。

Rule 6: Use appropriate tools to read the compiler's mind, and expect to be surprised by the code it produces. Inspect the code yourself before forming theories about what makes something faster or slower.

规则7：减少测量中的噪音。在安静的机器上运行您的基准测试，并运行几次，丢弃异常值。使用 -Xbatch 使用应用程序序列化编译器，并考虑设置 -XX：CICompilerCount = 1 以防止编译器与自己并行运行。尽量减少GC开销，设置 Xmx （足够大）等于 Xms 并使用 UseEpsilonGC 如果可用。

Rule 7: Reduce noise in your measurements. Run your benchmark on a quiet machine, and run it several times, discarding outliers. Use -Xbatch to serialize the compiler with the application, and consider setting -XX:CICompilerCount=1 to prevent the compiler from running in parallel with itself. Try your best to reduce GC overhead, set Xmx(large enough) equals Xms and use UseEpsilonGC if it is available.

规则8：使用库作为基准测试，因为它可能更有效，并且已经针对此唯一目的进行了调试。例如 JMH ， Caliper 或 Bill and保罗优秀的UCSD Java基准。

Rule 8: Use a library for your benchmark as it is probably more efficient and was already debugged for this sole purpose. Such as JMH, Caliper or Bill and Paul's Excellent UCSD Benchmarks for Java.

这篇关于如何在Java中编写正确的微基准测试？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！