如何使用testthat以未知顺序测试多个警告? [英] How to test for multiple warnings in an unknown order using testthat?

问题描述

我想测试一个函数在警告的顺序可以变化时会生成多个警告(4个或更多).我最好的尝试是基于前瞻性RegExp匹配.简化为2条警告，我知道我的RegExp在单个字符串输出上工作，因为以下两个都是正确的:

I want to test that a function generates multiple warnings (4 or more), when the order of the warnings can vary. My best attempt to do this is based on look-ahead RegExp matching. Simplifying to just 2 warnings, I know my RegExp work on a single string output, because both of the following are true:

grepl("(?s)(?=.*2)(?=.*1)", "* warn 1.\n* warn 2.", perl=TRUE)
grepl("(?s)(?=.*2)(?=.*1)", "* warn 2.\n* warn 1.", perl=TRUE)

但是，当使用testhat::expect_warning

# The function generating warnings:
foo <- function() { warning("warn 1."); warning("warn 2.") }
foo()
Warning messages:
1: In foo() : warn 1.
2: In foo() : warn 2.

# Testing it
expect_warning( foo(), "(?s)(?=.*1)(?=.*2)", perl=TRUE)

Error: foo() does not match '(?s)(?=.*1)(?=.*2)'. Actual values:
* warn 1.
* warn 2.

我怀疑这是因为expect_warning的内部在做类似分别针对每个警告测试给定RegExp的事情-为什么expect_warning( ... all=TRUE )参数可能有意义.

I suspect this is because the internals of expect_warning are doing something like testing the given RegExp against each warning separately--why an expect_warning( ... all=TRUE ) parameter might be meaningful.

不幸的是，我不能与"1 | 2"这样的RegExp一起使用；如果只给出一个警告，则成功.

Unfortunately I can't use this with a RegExp like "1 | 2"; that succeeds if only one warning is given.

我还希望避免多次运行该函数，并且每次都测试一个不同的警告.要测试真实功能，需要大量的设置和拆卸代码.它与文件系统有很大的交互作用，并且由于它是我正在测试的文件系统警告，因此我不能对此进行嘲笑.而且，我想在多种情况下测试警告，每种情况都需要不同的设置和拆卸代码，因此这会使我的测试迅速膨胀.

I also want to avoid running the function multiple times and testing a different warning each time. There is a large amount of setup and teardown code needed to test the real function. It interacts heavily with the file system, and since it is the file-system warnings I am testing for, I can't mock that. Moreover, I want to test for the warnings in multiple situations, each of which requires different setup and teardown code, so this quickly bloats my tests.

关于如何简单且一次地测试多个警告的任何建议吗?

Any suggestion on how to test for multiple warnings simply and all at once?

推荐答案

因此，我整理了一个解决方案，其中涉及使用我自己的警告捕获循环并以字符串形式检查警告.这不是一次全部"的解决方案，但考虑到复杂的函数调用，它至少是紧凑的.适用于示例函数foo()的代码如下:

So I have cobbled together a solution that involves using my own warning catch loop and checking the warnings as strings. Not an "all at once" solution, but it is at least compact given a complex function call. My code as it would apply to the example function foo() is the following:

gotWarnings= character(0)
withCallingHandlers({
   got <- foo()
   }, warning= function(e) {
      # Push warning onto vector in parent frame.
      gotWarnings <<- c(gotWarnings, conditionMessage(e))
      invokeRestart("muffleWarning")
})

# Ensure no unexpected warnings, 
expect_equal(length(gotWarnings), 2)

# Test that each warning I want is there
expect_true( any( grepl( "warn 1\\.", gotWarnings )))
expect_true( any( grepl( "warn 2\\.", gotWarnings )))

弄清楚警告后如何继续进行处理是困难的部分；捕获到第一个警告后，tryCatch退出. 此处和

Figuring out how to continue processing after catching a warning was the hard part; tryCatch exits after catching the first warning. Here and here helped me work this out. Perhaps this could be converted into an expect_all_warnings test that takes a vector of RegEx to match, maybe with all= TRUE meaning no warning can go unmatched. I'll hold off accepting this as an answer in case somebody posts a better one, or at least one that packages a solution like this nicely.

这篇关于如何使用testthat以未知顺序测试多个警告?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！