IntelliJ：使用previos运行数据间歇性地重新运行随机测试失败 [英] IntelliJ: rerun intermittently failing random test with previos run's data

问题描述

我有一个带有许多测试的JUnit测试类。为了增加场景覆盖率，我们测试中的一些数据是随机的，这意味着它可能在各个测试运行之间需要一系列值，例如：

I have a JUnit test class with a number of tests. To increase scenario coverage some data in our tests is randomized, meaning it may take a range of values between individual test runs, for example:

protected MonthlyAmountWithRemainder getMonetaryAmountMultipleOf(int multiplier) {
    BigDecimal monthly = randomBigDecimal(1000);
    BigDecimal multiply = new BigDecimal(multiplier);
    BigDecimal total = monthly.multiply(multiply);
    return new MonthlyAmountWithRemainder(total, monthly, ZERO);
}

你看到这个 randomBigDecimal（1000）？这可能会产生0到1000之间的任何值。我们还可以在测试中随机化日期和其他一些值。

Do you see this randomBigDecimal(1000)? that may generate any value between 0 and 1000. We can also randomize date and some other values in the test.

通常情况下，我们的测试运行得很好，但是一旦出现在蓝色的月亮中（我正在谈论的当前场景是一次大约50次），测试失败没有任何明显的原因。可以想象，这种罕见的故障使得无法调试测试用例以找出失败的原因或修复它。

Typically, our tests run just fine, but once in a blue moon (current scenario I'm talking about is once in about 50 times), a test fails without any apparent reason. As you can imagine, such rare failure makes it impossible to debug a test case to find out the reason for the failure or fix it.

因此，问题是：是否有可能捕获失败的测试运行中生成的数据并使用完全相同的测试数据重新运行测试，以便我可以调试失败的情况？换句话说，我想重新实现我以前失败的测试运行。可以实现吗？

So, the question is: is it possible to capture the data generated in the failed test run and re-run the test with exactly the same test data, so that I could debug the failing scenario? In other words, I would like to re-live my previous failed test run. Could that be achieved?

推荐答案

简单：使用种子来种子随机数发生器。可以从随机数生成器本身创建种子（这样您就可以为每次运行获得不同的随机值）。核心要点是记录使用的种子。

Simple: use a seed to seed the random number generator. That seed can be created from the random number generator itself (so that you get different random values for each run). The core point is to then log the seed that gets used.

因为那时失败的重复归结为不使用随机种子，而是导致测试失败的种子值。

Because then a "fail repro" boils down to not use a random seed, but that very seed value that caused the test to fail.

当然：工作需要让事情正确 - 你想确保你真正使用不同的种子运行每日测试。但是你也想确保修复种子是微不足道的，并导致确定性的结果。

Of course: work is required to get things "right" - you want to ensure that you really run the daily tests using different seeds. But you also want to make sure that fixating the seed is trivial, and leads to deterministic results.

或者，每当有随机数据进入时，你应该查看quickcheck基于测试的方法（参见此处例如）。进入这种方法需要一些思考，但它通常值得花时间。我们的想法是您指定生产代码的某些属性 - 然后框架生成随机数据并尝试伪造属性。非常好的部分：一旦框架找到了打破您的属性的方法，它就会开始搜索导致问题的最小示例。

Alternatively, whenever random data comes in, you should look into "quickcheck"-based testing approaches (see here for example). It takes a bit of thinking to get into that approach, but it is often worth the time. The idea is that you specify certain properties of your production code - and then the framework generates random data and tries to falsify the properties. And the really nice part: as soon as the framework finds a way to break your properties, it starts searching for a minimal example leading to the problem.

这篇关于IntelliJ：使用previos运行数据间歇性地重新运行随机测试失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！