Scalatest异步测试套件与最终和何时就绪(org.scalatest.concurrent) [英] Scalatest Asynchronous Test Suites vs Eventually and WhenReady (org.scalatest.concurrent)

问题描述

我正在尝试使用scalatest 异步测试套件，但是除了对设置超时时间以及不设置超时时间，我看不到测试套件实际上添加了什么.

I'm trying to use scalatest Asynchronous Test Suites, but aside from some restrictions over setting timeout and what not, I don't see what does the test suite actually add.

我想知道是否有人喜欢使用scalatest进行异步测试，可以快速解释一下异步测试套件和org.scalatest.concurrent之间的区别.异步测试套件实际上在org.scalatest.concurrent上增加了什么?一种方法比另一种更好吗?

I wonder if anyone versed into asynchronous testing with scalatest could quickly explain the differences between Asynchronous Test Suites and org.scalatest.concurrent. What do async test suites actually add over org.scalatest.concurrent? Is one approach better over the other?

推荐答案

我们比较了以下ScalaTest工具来测试返回Future s的代码:

We compare the following ScalaTest facilities for testing code that returns Futures:

• 异步样式特征，例如， ScalaFutures
• 最终

• Asynchronous style traits, for example, AsyncFlatSpec
• ScalaFutures
• Eventually

class AsyncSpec extends AsyncFlatSpec {
  ...
  Future(3).map { v => assert(v == 3) }
  ...
}

• 非阻塞
• 我们可以在Future完成之前断言，即返回Future[Assertion]而不是Assertion
• 线程安全
• 单线程串行执行上下文
• Futures按启动顺序并依次执行并完成
• 用于将任务排入测试主体的线程也用于随后执行任务
• 断言可以映射到Futures
• 无需在测试体内进行阻塞，即使用Await，whenReady
• 消除因线程饥饿而造成的剥脱
• 测试主体中的最后一个表达式必须为Future[Assertion]
• 在测试主体中不支持多个断言
• 不能在测试体内使用阻塞构造，因为它会因为等待而永久挂起测试 入队但从未开始任务
• non-blocking
• we can assert before Future completes, i.e., return Future[Assertion] instead of Assertion
• thread-safe
• single-threaded serial execution context
• Futures execute and complete in the order they are started and one after another
• the same thread that is used to enqueue tasks in the test body is also used to execute them afterwards
• Assertions can be mapped over Futures
• no need to block inside the test body, i.e., use Await, whenReady
• eliminates flakiness due to thread starvation
• last expression in the test body must be Future[Assertion]
• does not support multiple assertions in the test body
• cannot use blocking constructs inside the test body as it will hang the test forever because of waiting on enqueued but never started task

class ScalaFuturesSpec extends FlatSpec with ScalaFutures {
  ...
  whenReady(Future(3) { v => assert(v == 3) }
  ...
}

• 阻止
• 我们必须等待完成Future才能返回Assertion
• 不是线程安全的
• 可能与全局执行上下文scala.concurrent.ExecutionContext.Implicits.global一起使用，这是一个 多线程池以并行执行
• 在同一测试正文中支持多个断言
• 测试主体中的最后一个表达不必是Assertion
• blocking
• we must wait to complete the Future before we can return Assertion
• not thread-safe
• Likely to be used with global execution context scala.concurrent.ExecutionContext.Implicits.global which is a multi-threaded pool for parallel execution
• supports multiple assertions within the same test body
• last expression in the test body does not have to be Assertion

class EventuallySpec extends FlatSpec with Eventually {
  ...
  eventually { assert(Future(3).value.contains(Success(3))) }
  ...
}

• 更通用的功能，不仅用于Futures
• 这里的语义是重试按名称传递的任何类型的代码块，直到满足断言为止
• 在测试Futures时，很可能会使用全局执行上下文
• 主要用于集成测试，该测试针对具有不可预测响应时间的真实服务进行测试
• more general facility intended not just for Futures
• semantics here are that of retrying a block of code of any type passed in by-name until assertion is satisfied
• when testing Futures it is likely global execution context will be used
• intended primarily for integration testing where testing against real services with unpredictable response times

scalatest-async-testing-comparison 是一个示例 展示了两种执行模型的差异.

scalatest-async-testing-comparison is an example demonstrating the difference in two execution model.

给出以下测试体

    val f1 = Future {
      val tmp = mutableSharedState
      Thread.sleep(5000)
      println(s"Start Future1 with mutableSharedState=$tmp in thread=${Thread.currentThread}")
      mutableSharedState = tmp + 1
      println(s"Complete Future1 with mutableSharedState=$mutableSharedState")
    }

    val f2 = Future {
      val tmp = mutableSharedState
      println(s"Start Future2 with mutableSharedState=$tmp in thread=${Thread.currentThread}")
      mutableSharedState = tmp + 1
      println(s"Complete Future2 with mutableSharedState=$mutableSharedState")
    }

    for {
      _ <- f1
      _ <- f2
    } yield {
      assert(mutableSharedState == 2)
    }

让我们考虑将AsyncSpec的输出与ScalaFuturesSpec

let us consider the output of AsyncSpec against ScalaFuturesSpec

• testOnly示例.AsyncSpec:

• testOnly example.AsyncSpec:

Start Future1 with mutableSharedState=0 in thread=Thread[pool-11-thread-3-ScalaTest-running-AsyncSpec,5,main]
Complete Future1 with mutableSharedState=1
Start Future2 with mutableSharedState=1 in thread=Thread[pool-11-thread-3-ScalaTest-running-AsyncSpec,5,main]
Complete Future2 with mutableSharedState=2

• testOnly示例.ScalaFuturesSpec:

• testOnly example.ScalaFuturesSpec:

  Start Future2 with mutableSharedState=0 in thread=Thread[scala-execution-context-global-119,5,main]
  Complete Future2 with mutableSharedState=1
  Start Future1 with mutableSharedState=0 in thread=Thread[scala-execution-context-global-120,5,main]
  Complete Future1 with mutableSharedState=1
  

请注意在串行执行模型中如何使用相同的线程以及如何按顺序完成期货.另一方面， 在全局执行模型中，使用了不同的线程，并且在Future1之前完成了Future2，这导致 共享可变状态下的竞争状态，这又导致测试失败.

Note how in serial execution model same thread is used and Futures completed in order. On the other hand, in global execution model different threads were used, and Future2 completed before Future1, which caused race condition on the shared mutable state, which in turn made the test fail.

在单元测试中，我们应该使用模拟子系统，其中返回的Futures应该几乎立即完成，因此 在单元测试中不需要Eventually.因此，选择是在异步样式和ScalaFutures之间.主要区别 两者之间的区别在于前者与后者不同.如果可能的话，我们绝不应该阻止，所以我们 应该更喜欢AsyncFlatSpec之类的异步样式.执行模型还有更大的不同.异步样式 默认情况下，使用自定义串行执行模型，该模型在共享的可变状态下提供线程安全，这与全局变量不同 经常与ScalaFutures一起使用的线程池支持的执行模型.总之，我的建议是我们使用异步样式 特质，除非我们有充分的理由不这样做.

In unit tests we should use mocked subsystems where returned Futures should be completing near-instantly, so there is no need for Eventually in unit tests. Hence the choice is between async styles and ScalaFutures. The main difference between the two is that former is non-blocking unlike the latter. If possible, we should never block, so we should prefer async styles like AsyncFlatSpec. Further big difference is the execution model. Async styles by default use custom serial execution model which provides thread-safety on shared mutable state, unlike global thread-pool backed execution model often used with ScalaFutures. In conclusion, my suggestion is we use async style traits unless we have a good reason not to.

这篇关于Scalatest异步测试套件与最终和何时就绪(org.scalatest.concurrent)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！