秒表基准测试可以接受吗？ [英] Is stopwatch benchmarking acceptable?

问题描述

有没有人使用秒表基准测试，还是应该始终使用性能工具？有没有可用于Java的免费工具？您使用了哪些工具？

为了澄清我的担忧，秒表基准测试会因操作系统调度而出错。在程序的给定运行中，操作系统可能会在您正在计时的函数中间安排另一个（或几个）进程。在Java中，如果你试图计算一个线程化的应用程序，事情会更糟糕，因为JVM调度程序会在混合中投入更多的随机性。

如何在基准测试时解决操作系统调度问题？

解决方案

如果您测量到足够有意义的迭代次数，则秒表基准测试很好。通常，我需要一些单位数秒的总经过时间。否则，您的结果很容易受到调度以及您的流程的其他操作系统中断的影响。

为此，我使用了一些静态方法，我构建了一个长的很久以前，这是基于 System.currentTimeMillis（）。

对于我使用过的分析工作 jProfiler 多年来一直发现它非常好。我最近查看了 YourKit ，这在WebSite上看起来很棒，但我根本没用过，个人。

为了回答有关安排中断的问题，我发现重复运行直到实现/观察到一致性，在实践中可以清除过程调度的异常结果。我还发现线程调度对5到30秒之间的运行没有实际影响。最后，根据我的经验，在经过几秒钟后，阈值调度对结果的影响可以忽略不计 - 我发现5秒运行的时间和迭代的运行时间平均为5分钟。

您可能还需要考虑预先运行测试代码大约10,000次来预热JIT，具体取决于您希望测试代码在现实生活中随时间运行的次数。 / p>

Does anyone ever use stopwatch benchmarking, or should a performance tool always be used? Are there any good free tools available for Java? What tools do you use?

To clarify my concerns, stopwatch benchmarking is subject to error due to operating system scheduling. On a given run of your program the OS might schedule another process (or several) in the middle of the function you're timing. In Java things are even a little bit worse if you're trying to time a threaded application, as the JVM scheduler throws even a little bit more randomness into the mix.

How do you address operating system scheduling when benchmarking?

解决方案

Stopwatch benchmarking is fine, provided you measure enough iterations to be meaningful. Typically, I require a total elapsed time of some number of single digit seconds. Otherwise, your results are easily significantly skewed by scheduling, and other O/S interruptions to your process.

For this I use a little set of static methods I built a long time ago, which are based on System.currentTimeMillis().

For the profiling work I have used jProfiler for a number of years and have found it very good. I have recently looked over YourKit, which seems great from the WebSite, but I've not used it at all, personally.

To answer the question on scheduling interruptions, I find that doing repeated runs until consistency is achieved/observed works in practice to weed out anomalous results from process scheduling. I also find that thread scheduling has no practical impact for runs of between 5 and 30 seconds. Lastly, after you pass the few seconds threshold scheduling has, in my experience, negligible impact on the results - I find that a 5 second run consistently averages out the same as a 5 minute run for time/iteration.

You may also want to consider prerunning the tested code about 10,000 times to "warm up" the JIT, depending on the number of times you expect the tested code to run over time in real life.

这篇关于秒表基准测试可以接受吗？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！