Scalacheck 不会正确报告失败的情况 [英] Scalacheck won't properly report the failing case

问题描述

我已经编写了以下规范

"An IP4 address" should "belong to just one class" in {
    val addrs = for {
        a <- Gen.choose(0, 255)
        b <- Gen.choose(0, 255)
        c <- Gen.choose(0, 255)
        d <- Gen.choose(0, 255)
    } yield s"$a.$b.$c.$d"

    forAll (addrs) { ip4s =>
        var c: Int = 0
        if (IP4_ClassA.unapply(ip4s).isDefined) c = c + 1
        if (IP4_ClassB.unapply(ip4s).isDefined) c = c + 1
        if (IP4_ClassC.unapply(ip4s).isDefined) c = c + 1
        if (IP4_ClassD.unapply(ip4s).isDefined) c = c + 1
        if (IP4_ClassE.unapply(ip4s).isDefined) c = c + 1
        c should be (1)
    }
}

范围很明确.

测试成功通过但当我强制它失败时(例如通过注释掉 if 语句之一)然后 ScalaCheck 正确报告错误但消息没有正确提及实际值用于评估命题.更具体地说，我得到:

The test passes successfully but when I force it to fail (for example by commenting out one of the if statements) then ScalaCheck correctly reports the error but the message doesn't mention correctly the actual value used to evaluate the proposition. More specifically I get:

[info] An IP4 address
[info] - should belong to just one class *** FAILED ***
[info]   TestFailedException was thrown during property evaluation.
[info]     Message: 0 was not equal to 1
[info]     Location: (NetSpec.scala:105)
[info]     Occurred when passed generated values (
[info]       arg0 = "" // 4 shrinks
[info]     )

在哪里可以看到 arg0 = ""//4 缩小 不显示值.

where you can see arg0 = "" // 4 shrinks doesn't show the value.

我什至尝试添加一个简单的 println 语句来查看案例，但输出似乎被修剪了.我得到了这样的东西

I've tried to add even a simple println statement to review the cases but the output appears to be trimmed. I get something like this

192.168.0.1
189.168.
189.
1

解决方案

import org.scalacheck.Prop.forAllNoShrink
import org.scalatest.prop.Checkers.check

"An IP4 address" should "belong to just one class" in {
  val addrs = for {
    a <- Gen.choose(0, 255)
    b <- Gen.choose(0, 255)
    c <- Gen.choose(0, 255)
    d <- Gen.choose(0, 255)
  } yield s"$a.$b.$c.$d"
  check {
    forAllNoShrink(addrs) { ip4s =>
      var c: Int = 0
      if (IP4.ClassA.unapply(ip4s).isDefined) c = c + 1
      if (IP4.ClassB.unapply(ip4s).isDefined) c = c + 1
      if (IP4.ClassC.unapply(ip4s).isDefined) c = c + 1
      if (IP4.ClassD.unapply(ip4s).isDefined) c = c + 1
      if (IP4.ClassE.unapply(ip4s).isDefined) c = c + 1
      c == (1)
    }
  }
}

推荐答案

这是由ScalaCheck 的测试用例简化功能引起的.ScalaCheck 只是看到你的生成器产生了一个字符串值.每当它找到一个使您的属性为假的值时，它都会尝试简化该值.在你的情况下，它简化了四次，直到它以一个空字符串结束，这仍然使你的属性为假.

This is caused by ScalaCheck's test case simplification feature. ScalaCheck just sees that your generator produces a string value. Whenever it finds a value that makes your property false, it tries to simplify that value. In your case, it simplifies it four times until it ends up with an empty string, that still makes your property false.

所以这是预期的，虽然令人困惑，行为.但是您可以通过三种不同的方式改善这种情况.

So this is expected, although confusing, behavior. But you can improve the situation in three different ways.

您可以选择其他数据结构来表示您的 IP 地址.这将使 ScalaCheck 能够以更智能的方式简化您的测试用例.例如，使用以下生成器:

You can select another data structure to represent your IP addresses. This will make ScalaCheck able to simplify your test cases in a more intelligent way. For example, use the following generator:

val addrs = Gen.listOfN(4, Gen.choose(0,255))

现在 ScalaCheck 知道您的生成器只生成长度为 4 的列表，并且它只包含 0 到 255 之间的数字.测试用例简化过程将考虑到这一点，不会创建任何无法生成的值发电机从一开始.您可以改为在属性内转换为字符串.

Now ScalaCheck knows that your generator only produces lists of length 4, and that it only contains numbers between 0 and 255. The test case simplification process will take this into account and not create any values that couldn't have been produced by the generator from start. You can do the conversion to string inside your property instead.

第二种方法是直接向生成器添加一个过滤器，它告诉 ScalaCheck IP 地址字符串应该是什么样子.此过滤器在测试用例简化期间使用.定义一个检查有效字符串的函数，并以这种方式将其附加到现有生成器:

A second method is to add a filter directly to your generator, which tells ScalaCheck how an IP address string should look like. This filter is used during test case simplification. Define a function that checks for valid strings and attach it to your existing generator this way:

def validIP(ip: String): Boolean = ...

val validAddrs = addrs.suchThat(validIP)

forAll(validAddrs) { ... }

第三种方法是完全禁用测试用例简化功能，使用 forAllNoShrink 而不是 forAll:

The third method is to simply disable the test case simplification feature altogether by using forAllNoShrink instead of forAll:

Prop.forAllNoShrink(addrs) { ... }

我还应该提到，前两种方法需要 ScalaCheck 版本 >= 1.11.0 才能正常运行.

I should also mention that the two first methods require ScalaCheck version >= 1.11.0 to function properly.

更新:

由于 https://github.com/rickynils/scalacheck/issues/89.希望这可以在未来版本的 ScalaCheck 中得到修复.

The listOfN list length is actually not respected by the shrinker any more, due to https://github.com/rickynils/scalacheck/issues/89. Hopefully this can be fixed in a future version of ScalaCheck.

这篇关于Scalacheck 不会正确报告失败的情况的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！