Scalacheck无法正确报告失败案例 [英] Scalacheck won't properly report the failing case

问题描述

我写了以下规范

"An IP4 address" should "belong to just one class" in {
    val addrs = for {
        a <- Gen.choose(0, 255)
        b <- Gen.choose(0, 255)
        c <- Gen.choose(0, 255)
        d <- Gen.choose(0, 255)
    } yield s"$a.$b.$c.$d"

    forAll (addrs) { ip4s =>
        var c: Int = 0
        if (IP4_ClassA.unapply(ip4s).isDefined) c = c + 1
        if (IP4_ClassB.unapply(ip4s).isDefined) c = c + 1
        if (IP4_ClassC.unapply(ip4s).isDefined) c = c + 1
        if (IP4_ClassD.unapply(ip4s).isDefined) c = c + 1
        if (IP4_ClassE.unapply(ip4s).isDefined) c = c + 1
        c should be (1)
    }
}

在范围上很清楚.

测试成功通过，但是当我强制测试失败(例如，通过注释掉if语句之一)时，ScalaCheck会正确报告错误，但消息未正确提及用于评估命题的实际值.更具体地说，我得到:

The test passes successfully but when I force it to fail (for example by commenting out one of the if statements) then ScalaCheck correctly reports the error but the message doesn't mention correctly the actual value used to evaluate the proposition. More specifically I get:

[info] An IP4 address
[info] - should belong to just one class *** FAILED ***
[info]   TestFailedException was thrown during property evaluation.
[info]     Message: 0 was not equal to 1
[info]     Location: (NetSpec.scala:105)
[info]     Occurred when passed generated values (
[info]       arg0 = "" // 4 shrinks
[info]     )

可以看到arg0 = "" // 4 shrinks的地方没有显示该值.

where you can see arg0 = "" // 4 shrinks doesn't show the value.

我试图甚至添加一个简单的println语句来审查案例，但输出似乎已被裁剪.我得到这样的东西

I've tried to add even a simple println statement to review the cases but the output appears to be trimmed. I get something like this

192.168.0.1
189.168.
189.
1

解决方案

import org.scalacheck.Prop.forAllNoShrink
import org.scalatest.prop.Checkers.check

"An IP4 address" should "belong to just one class" in {
  val addrs = for {
    a <- Gen.choose(0, 255)
    b <- Gen.choose(0, 255)
    c <- Gen.choose(0, 255)
    d <- Gen.choose(0, 255)
  } yield s"$a.$b.$c.$d"
  check {
    forAllNoShrink(addrs) { ip4s =>
      var c: Int = 0
      if (IP4.ClassA.unapply(ip4s).isDefined) c = c + 1
      if (IP4.ClassB.unapply(ip4s).isDefined) c = c + 1
      if (IP4.ClassC.unapply(ip4s).isDefined) c = c + 1
      if (IP4.ClassD.unapply(ip4s).isDefined) c = c + 1
      if (IP4.ClassE.unapply(ip4s).isDefined) c = c + 1
      c == (1)
    }
  }
}

推荐答案

这是由ScalaCheck的测试用例简化功能引起的. ScalaCheck只是看到您的生成器产生了一个字符串值.只要找到使您的属性为假的值，它就会尝试简化该值.在您的情况下，它会简化四次，直到最后得到一个空字符串，这仍然会使您的媒体资源为假.

This is caused by ScalaCheck's test case simplification feature. ScalaCheck just sees that your generator produces a string value. Whenever it finds a value that makes your property false, it tries to simplify that value. In your case, it simplifies it four times until it ends up with an empty string, that still makes your property false.

因此，这虽然令人困惑，但却是预期的行为.但是您可以通过三种不同的方式来改善这种情况.

So this is expected, although confusing, behavior. But you can improve the situation in three different ways.

您可以选择另一个数据结构来表示您的IP地址.这将使ScalaCheck能够以更智能的方式简化您的测试用例.例如，使用以下生成器:

You can select another data structure to represent your IP addresses. This will make ScalaCheck able to simplify your test cases in a more intelligent way. For example, use the following generator:

val addrs = Gen.listOfN(4, Gen.choose(0,255))

现在，ScalaCheck知道您的生成器仅生成长度为4的列表，并且它仅包含0到255之间的数字.测试用例的简化过程将考虑到这一点，并且不会创建任何不能由生成的值从一开始就生成发电机.您可以改为在属性内将其转换为字符串.

Now ScalaCheck knows that your generator only produces lists of length 4, and that it only contains numbers between 0 and 255. The test case simplification process will take this into account and not create any values that couldn't have been produced by the generator from start. You can do the conversion to string inside your property instead.

第二种方法是直接将过滤器添加到生成器，该过滤器告诉ScalaCheck IP地址字符串的外观.在简化测试用例期间使用此过滤器.定义一个检查有效字符串的函数，并通过以下方式将其附加到您现有的生成器上:

A second method is to add a filter directly to your generator, which tells ScalaCheck how an IP address string should look like. This filter is used during test case simplification. Define a function that checks for valid strings and attach it to your existing generator this way:

def validIP(ip: String): Boolean = ...

val validAddrs = addrs.suchThat(validIP)

forAll(validAddrs) { ... }

第三种方法是使用forAllNoShrink而不是forAll来完全禁用测试用例的简化功能:

The third method is to simply disable the test case simplification feature altogether by using forAllNoShrink instead of forAll:

Prop.forAllNoShrink(addrs) { ... }

我还应该提到，前两种方法要求ScalaCheck版本> = 1.11.0才能正常运行.

I should also mention that the two first methods require ScalaCheck version >= 1.11.0 to function properly.

更新:

由于>://github.com/rickynils/scalacheck/issues/89 .希望可以在以后的ScalaCheck版本中解决此问题.

The listOfN list length is actually not respected by the shrinker any more, due to https://github.com/rickynils/scalacheck/issues/89. Hopefully this can be fixed in a future version of ScalaCheck.

这篇关于Scalacheck无法正确报告失败案例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！