用googletest进行基准测试? [英] Benchmarking with googletest?
问题描述
我有一个运行三种状态的模拟器:
I have a simulator that runs through three states:
- 单线程启动(I/O正常)
- 与内存绑定的多线程模拟阶段(I/O不好)
- 模拟后，加入后单线程阶段(I/O正常)
该死的！在标准测试期间，CPU使用率从 100％下降到20％，并且总运行时间比正常情况下长了30倍左右(130秒对4.2秒).
What the heck! During standard testing, CPU usage dropped from 100% down to 20%, and the total run took about 30 times longer than normal (130secs vs 4.2secs).
当Callgrind没有发现任何可疑内容时，我的脑袋嗡嗡作响，因为我正处于回滚到最后一次提交的悬崖上，丢失了所有错误修复程序.
When Callgrind revealed nothing suspicious, my head buzzed as I was on the precipice of rolling back to the last commit, losing all bug-fixes.
心灰意冷，我在奔跑过程中走进服务器机房，发现讨厌的磨擦声，后来证实是由于写入
Discouraged, I walked into the server room during a run and noticed nasty grinding sounds, verified later to be caused by writes to Mysql sockets in /proc/PID/fd!!! It turned out that Mysql code, several layers deep in Stage 2., was causing problems.
经验教训
- 意外的I/O可能会对实时应用程序造成致命影响
- 单元测试还不够:我也需要基准测试
修复，我将在ReadAllowed()和WriteAllowed()上引入线程本地存储IOSentinel和asserts()，以确保第2阶段线程永远不会做任何IO.
Fix I will introduce thread-local-storage IOSentinels and asserts() on ReadAllowed() and WriteAllowed() to ensure that Stage 2 threads will never do any IO.
有人通过googletest附加/编写基准测试框架有运气吗?
很不幸，这次我所有的googletest测试都通过了.如果我退后一会儿而又不注意运行时间，那将是一次灾难性的提交，并且可能很难解决.
Unfortunately, all my googletests passed this time. Had I stepped away for a bit and come back without noticing the run-time, this would have been a disastrous commit, and possibly much harder to fix.
如果运行时间超过上次运行时间的2或3倍，我希望googletest失败:这最后一部分很棘手，因为对于非常快速的运行，系统状态可能会花费两倍的时间，但仍然可以.但是对于长时间的模拟运行/测试，我不希望运行时发生很大的变化(> 50％的情况是不寻常的).
I would like googletest to fail if a run takes >2 or 3 times the last runtime: this last part is tricky because for very quick runs, system state can cause something to take twice as long but still be ok. But for a long simulation run/test, I don't expect runtimes to change by a great deal (>50% would be unusual).
我愿意接受这里的建议，但是最好是进行自动维护的低维护检查，这样即使系统的输出看起来正常，如果系统突然变慢，也很明显.
I am open to suggestions here, but it would be nice to have a low-maintenance check that would work with automated testing so it will be obvious if the system suddenly got slow, even if all the outputs appear to be ok.
推荐答案
对此问题进行了一些更新(2016年):
Some updates on this question (in 2016):
Here is a nice blog-post of Nick Brunn about his Hayai benchmarking framework. (2012)
- 它不提供指定运行时间要求的可能性.
- 它与Google Test非常相似.语法等
- 它向用户或Continuous Integration框架提供基准测试结果.还可以查看 MojaveWastelander 的活动开发和MSVC支持分支.
- It does not provide the possibility of specifying running time requirements.
- It is very similar to Google Test. Syntax, etc.
- It provides the benchmarking results to the user or a Continous Integration framework. Also have a look at MojaveWastelander's fork for active development and MSVC support.
Google published 'Benchmark' in 2014. This provides similar behaviour then Hayai above. As far as I understand, defining requirements is not possible. Again, the syntax is inspired by GoogleTest.
- 甚至还有一些高级功能，例如测量复杂度(big-O).
GoogleTest在 Github 上将其作为开放功能.基本的实现，但它还不是GoogleTest的一部分./p>
GoogleTest has this as an open feature on Github. There is a rudimentary implementation but it is not part of GoogleTest yet.
这篇关于用googletest进行基准测试?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
