使用随机输入的最佳实践进行测试 [英] Testing with random inputs best practices
问题描述
注意:我将以下几段作为背景.如果您只想使用TL; DR,请随意跳至已编号的问题,因为它们仅与该信息间接相关.
NOTE: I mention the next couple of paragraphs as background. If you just want a TL;DR, feel free to skip down to the numbered questions as they are only indirectly related to this info.
我目前正在编写一个python脚本,该脚本可以处理POSIX日期(除其他事项外).但是,对这些进行单元测试似乎有些困难,因为可能会遇到各种各样的日期和时间.
I'm currently writing a python script that does some stuff with POSIX dates (among other things). Unit testing these seems a little bit difficult though, since there's such a wide range of dates and times that can be encountered.
当然,对每个可能的日期/时间组合进行测试都是不切实际的,因此,我认为我将尝试进行单元测试,该测试将输入随机化,然后在测试失败时报告输入的内容.从本质上讲,我认为如果尝试运行所有潜在的问题区域(由于缺少东西)或测试所有案例(由于完全不可行),我的测试的完整性要比我想做的要高得多足够的时间.
Of course, it's impractical for me to try to test every single date/time combination possible, so I think I'm going to try a unit test that randomizes the inputs and then reports what the inputs were if the test failed. Statisically speaking, I figure that I can achieve a bit more completeness of testing than I could if I tried to think of all potential problem areas (due to missing things) or testing all cases (due to sheer infeasability), assuming that I run it enough times.
因此,这里有几个问题(主要与上述问题间接相关):
So here are a few questions (mainly indirectly related to the above ):
- 哪种类型的代码适合进行随机测试?不是什么类型的代码?
- 如何确定使用随机输入运行代码的次数?我之所以这样问,是因为我希望有足够大的样本来确定任何错误,但不想等待一个星期才能得到我的结果.
- 这些测试是否非常适合单元测试,或者是否有另一种测试可以很好地配合使用?
- 做这种事情还有其他最佳实践吗?
- What types of code are good candidates for randomized testing? What types of code aren't?
- How do I go about determining the number of times to run the code with randomized inputs? I ask this because I want to have a large enough sample to determine any bugs, but don't want to wait a week to get my results.
- Are these kinds of tests well suited for unit tests, or is there another kind of test that it works well with?
- Are there any other best practices for doing this kind of thing?
相关主题:
- 单元测试中的随机数据?
- Random data in unit tests?
- 请确保您对1900年至2100年之间每年的每个月的开始,中间和结尾进行测试(当然,如果这些测试适合您的代码).
- 使用多种文化,如果已知,则使用所有文化".
- 尝试第0天"和每个月结束后的一天"等.
Related topics:
推荐答案
我同意Federico-随机测试会适得其反.如果测试无法可靠地通过或失败,则很难对其进行修复并知道已修复. (当然,当您引入不可靠的依赖关系时,这也是一个问题.)
I agree with Federico - randomised testing is counterproductive. If a test won't reliably pass or fail, it's very hard to fix it and know it's fixed. (This is also a problem when you introduce an unreliable dependency, of course.)
但是,您可能想确保以其他方式具有良好的数据覆盖率.例如:
Instead, however, you might like to make sure you've got good data coverage in other ways. For instance:
简而言之,仍然可以尝试许多值,但是可以通过编程和可重复的方式进行.您不需要将所有要成为文字的值都当作测试中的文字-可以在测试的一个轴上循环所有已知值,等等.
In short, still try a lot of values, but do so programmatically and repeatably. You don't need every value you try to be a literal in a test - it's fine to loop round all known values for one axis of your testing, etc.
您永远不会获得完整的报道,但至少可以重复.
You'll never get complete coverage, but it will at least be repeatable.
我确定在 个地方,随机测试很有用,尽管可能不适用于单元测试.但是,在这种情况下,我想提出一些建议:使用一个RNG创建一个随机但已知的种子,然后使用该值播种一个新的RNG-并将其记录下来.这样,如果发生有趣的事情,您 将能够通过使用记录的种子启动RNG来重现它.
I'm sure there are places where random tests are useful, although probably not for unit tests. However, in this case I'd like to suggest something: use one RNG to create a random but known seed, and then seed a new RNG with that value - and log it. That way if something interesting happens you will be able to reproduce it by starting an RNG with the logged seed.
这篇关于使用随机输入的最佳实践进行测试的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!