将单元测试标记为JUnit中的预期故障 [英] Mark unit test as an expected failure in JUnit

问题描述

如何将测试标记为JUnit 4中的预期失败？

How can I mark a test as an expected failure in JUnit 4?

在这种情况下，我想继续运行此测试，直到上游修补了某些内容。忽略测试有点太过分了，因为那时我可能会忘记它。我可以添加 @expected 注释并捕获 assertThat 引发的异常，但这似乎也在于关于预期的行为。

In this case I want to continue to run this test until something is patched upstream. Ignoring the test goes a little too far, as then I might forget about it. I may be able to add an @expected annotation and catch the exception thrown by assertThat, but that also seems to lie about the expected behavior.

以下是我目前的测试结果：

Here's what my current test looks like:

@Test
public void unmarshalledDocumentHasExpectedValue() 
{
    doc = unmarshaller.unmarshal(getResourceAsStream("mydoc.xml"));
    final ST title = doc.getTitle();
    assertThat(doc.getTitle().toStringContent(), equalTo("Expected"));
}

该断言应该成功，但由于上游错误，它不会。然而，那个测试是正确的;它应该成功。事实上，我发现的所有替代方案都具有误导性。现在我认为 @Ignore（这个测试应该在固定上游时通过）是我最好的选择，但我仍然要记得回到它。我更喜欢测试运行。

That assert should succeed, but because of an upstream bug it doesn't. Yet, that test is correct; it should succeed. Virtually all the alternatives that I've found are misleading. Right now I think @Ignore("This test should pass once fixed upstream") is my best bet, but I still have to remember to come back to it. I'd prefer that the test run.

在Python中我可以使用 expectedFailure decorator：

In Python I can use the expectedFailure decorator:

class ExpectedFailureTestCase(unittest.TestCase):
    @unittest.expectedFailure
    def test_fail(self):
        self.assertEqual(1, 0, "broken")

使用Qt的 Q ++中的QTestLib ，您可以使用 QEXPECT_FAIL ：

With Qt's QTestLib in C++, you can use QEXPECT_FAIL:

QEXPECT_FAIL("", "Will be fixed next version", Continue);
QCOMPARE(i, 42);

在上述两种情况下，单位测试都会运行，这是我希望发生的。我在JUnit中遗漏了什么？

In both cases above, the unit test runs which is what I'm hoping to have happen. Am I missing something in JUnit?

推荐答案

我假设您希望测试通过，如果您的断言失败，但是如果断言成功，那么测试也应该通过。

I'm assuming here that you want the test to pass if your assert fails, but if the assert succeeds, then the test should pass as well.

最简单的方法是使用 TestRule 。 TestRule提供了在运行测试方法之前和之后执行代码的机会。下面是一个示例：

The easiest way to do this is to use a TestRule. TestRule gives the opportunity to execute code before and after a test method is run. Here is an example:

public class ExpectedFailureTest {
    public class ExpectedFailure implements TestRule {
        public Statement apply(Statement base, Description description) {
            return statement(base, description);
        }

        private Statement statement(final Statement base, final Description description) {
            return new Statement() {
                @Override
                public void evaluate() throws Throwable {
                    try {
                        base.evaluate();
                    } catch (Throwable e) {
                        if (description.getAnnotation(Deprecated.class) != null) {
                            // you can do whatever you like here.
                            System.err.println("test failed, but that's ok:");
                        } else {
                            throw e;
                        }
                    }
                }
            };
        }
    }

    @Rule public ExpectedFailure expectedFailure = new ExpectedFailure();

    // actually fails, but we catch the exception and make the test pass.
    @Deprecated
    @Test public void testExpectedFailure() {
        Object o = null;
        o.equals("foo");
    }

    // fails
    @Test public void testExpectedFailure2() {
        Object o = null;
        o.equals("foo");
    }
}

首先，请注意第一种方法标记为 @Deprecated 。我正在使用它作为我想忽略任何断言失败的方法的标记。你可以做任何你喜欢的方法来确定方法，这只是一个例子。

First, note that the first method is marked as @Deprecated. I'm using this as a marker for the method for which I want to ignore any assertion failures. You can do whatever you like to identify the methods, this is just an example.

接下来，在 ExpectedFailure＃apply（），当我执行base.evaluate（）时，我正在捕获任何Throwable（包括AssertionError），如果方法用注释@Deprecated标记，我会忽略错误。您可以根据版本号，某些文本等执行您喜欢的任何逻辑来决定是否应该忽略错误。您还可以将动态确定的标志传递给ExpectedFailure以允许它对某些版本号失败：

Next, in the ExpectedFailure#apply(), when I do the base.evaluate(), I'm catching any Throwable (which includes AssertionError) and if the method is marked with the annotation @Deprecated, I ignore the error. You can perform whatever logic you like to decide whether you should ignore the error or not, based on version number, some text, etc. You can also pass a dynamically determined flag into ExpectedFailure to allow it to fail for certain version numbers:

public void unmarshalledDocumentHasExpectedValue() {
    doc = unmarshaller.unmarshal(getResourceAsStream("mydoc.xml"));

    expectedFailure.setExpectedFailure(doc.getVersionNumber() < 3000);

    final ST title = doc.getTitle();
    assertThat(doc.getTitle().toStringContent(), equalTo("Expected"));
}

有关更多示例，请参阅 ExternalResource 和 ExpectedException

For further examples, see ExternalResource, and ExpectedException

如果你想把测试标记为忽略而不是成功，它会变得有点复杂，因为测试在执行之前会被忽略，所以你必须回顾一下将测试标记为忽略，这将涉及构建自己的Runner。为了给您一个开始，请参阅我对的回答如何在套件中定义JUnit方法规则？。或者问另一个问题。

If you want to mark you tests as Ignored rather than Success, it becomes a bit more complex, because tests are ignored before they are executed, so you have to retrospectively mark a test as ignored, which would involve constructing your own Runner. To give you a start, see my answer to How to define JUnit method rule in a suite?. Or ask another question.

这篇关于将单元测试标记为JUnit中的预期故障的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！