使用随机输入的最佳实践进行测试 [英] Testing with random inputs best practices

问题描述

注意:我将以下几段作为背景.如果您只想使用TL; DR，请随意跳至已编号的问题，因为它们仅与该信息间接相关.

NOTE: I mention the next couple of paragraphs as background. If you just want a TL;DR, feel free to skip down to the numbered questions as they are only indirectly related to this info.

我目前正在编写一个python脚本，该脚本可以处理POSIX日期(除其他事项外).但是，对这些进行单元测试似乎有些困难，因为可能会遇到各种各样的日期和时间.

I'm currently writing a python script that does some stuff with POSIX dates (among other things). Unit testing these seems a little bit difficult though, since there's such a wide range of dates and times that can be encountered.

当然，对每个可能的日期/时间组合进行测试都是不切实际的，因此，我认为我将尝试进行单元测试，该测试将输入随机化，然后在测试失败时报告输入的内容.从本质上讲，我认为如果尝试运行所有潜在的问题区域(由于缺少东西)或测试所有案例(由于完全不可行)，我的测试的完整性要比我想做的要高得多足够的时间.

Of course, it's impractical for me to try to test every single date/time combination possible, so I think I'm going to try a unit test that randomizes the inputs and then reports what the inputs were if the test failed. Statisically speaking, I figure that I can achieve a bit more completeness of testing than I could if I tried to think of all potential problem areas (due to missing things) or testing all cases (due to sheer infeasability), assuming that I run it enough times.

因此，这里有几个问题(主要与上述问题间接相关):

So here are a few questions (mainly indirectly related to the above ):

1. 哪种类型的代码适合进行随机测试?不是什么类型的代码?
   • 如何确定使用随机输入运行代码的次数?我之所以这样问，是因为我希望有足够大的样本来确定任何错误，但不想等待一个星期才能得到我的结果.
   • 这些测试是否非常适合单元测试，或者是否有另一种测试可以很好地配合使用?
   • 做这种事情还有其他最佳实践吗?

1. What types of code are good candidates for randomized testing? What types of code aren't?
   • How do I go about determining the number of times to run the code with randomized inputs? I ask this because I want to have a large enough sample to determine any bugs, but don't want to wait a week to get my results.
   • Are these kinds of tests well suited for unit tests, or is there another kind of test that it works well with?
   • Are there any other best practices for doing this kind of thing?

相关主题:

• 单元测试中的随机数据?

Related topics:

• Random data in unit tests?

推荐答案

我同意Federico-随机测试会适得其反.如果测试无法可靠地通过或失败，则很难对其进行修复并知道已修复. (当然，当您引入不可靠的依赖关系时，这也是一个问题.)

I agree with Federico - randomised testing is counterproductive. If a test won't reliably pass or fail, it's very hard to fix it and know it's fixed. (This is also a problem when you introduce an unreliable dependency, of course.)

但是，您可能想确保以其他方式具有良好的数据覆盖率.例如:

Instead, however, you might like to make sure you've got good data coverage in other ways. For instance:

• 请确保您对1900年至2100年之间每年的每个月的开始，中间和结尾进行测试(当然，如果这些测试适合您的代码).
• 使用多种文化，如果已知，则使用所有文化".
• 尝试第0天"和每个月结束后的一天"等.

简而言之，仍然可以尝试许多值，但是可以通过编程和可重复的方式进行.您不需要将所有要成为文字的值都当作测试中的文字-可以在测试的一个轴上循环所有已知值，等等.

In short, still try a lot of values, but do so programmatically and repeatably. You don't need every value you try to be a literal in a test - it's fine to loop round all known values for one axis of your testing, etc.

您永远不会获得完整的报道，但至少可以重复.

You'll never get complete coverage, but it will at least be repeatable.

我确定在 个地方，随机测试很有用，尽管可能不适用于单元测试.但是，在这种情况下，我想提出一些建议:使用一个RNG创建一个随机但已知的种子，然后使用该值播种一个新的RNG-并将其记录下来.这样，如果发生有趣的事情，您 将能够通过使用记录的种子启动RNG来重现它.

I'm sure there are places where random tests are useful, although probably not for unit tests. However, in this case I'd like to suggest something: use one RNG to create a random but known seed, and then seed a new RNG with that value - and log it. That way if something interesting happens you will be able to reproduce it by starting an RNG with the logged seed.

这篇关于使用随机输入的最佳实践进行测试的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！