将单元测试标记为 JUnit 中的预期失败 [英] Mark unit test as an expected failure in JUnit

问题描述

如何在 JUnit 4 中将测试标记为预期失败?

How can I mark a test as an expected failure in JUnit 4?

在这种情况下，我想继续运行此测试，直到上游修补某些内容为止.忽略测试有点过头了，因为那样我可能会忘记它.我也许可以添加一个 @expected 注释并捕获 assertThat 抛出的异常，但这似乎也与预期的行为有关.

In this case I want to continue to run this test until something is patched upstream. Ignoring the test goes a little too far, as then I might forget about it. I may be able to add an @expected annotation and catch the exception thrown by assertThat, but that also seems to lie about the expected behavior.

这是我当前的测试结果:

Here's what my current test looks like:

@Test
public void unmarshalledDocumentHasExpectedValue() 
{
    doc = unmarshaller.unmarshal(getResourceAsStream("mydoc.xml"));
    final ST title = doc.getTitle();
    assertThat(doc.getTitle().toStringContent(), equalTo("Expected"));
}

该断言应该成功，但由于上游错误，它没有成功.然而，那个测试是正确的；它应该成功.我发现的几乎所有替代方案都具有误导性.现在我认为 @Ignore("This test should pass once once fix upstream") 是我最好的选择，但我仍然要记得回到它.我更喜欢测试运行.

That assert should succeed, but because of an upstream bug it doesn't. Yet, that test is correct; it should succeed. Virtually all the alternatives that I've found are misleading. Right now I think @Ignore("This test should pass once fixed upstream") is my best bet, but I still have to remember to come back to it. I'd prefer that the test run.

在 Python 中，我可以使用 expectedFailure 装饰器:

In Python I can use the expectedFailure decorator:

class ExpectedFailureTestCase(unittest.TestCase):
    @unittest.expectedFailure
    def test_fail(self):
        self.assertEqual(1, 0, "broken")

使用 Qt 的 QTestLib 在 C++ 中，您可以使用 QEXPECT_FAIL:

With Qt's QTestLib in C++, you can use QEXPECT_FAIL:

QEXPECT_FAIL("", "Will be fixed next version", Continue);
QCOMPARE(i, 42);

在上述两种情况下，单元测试都会运行，这正是我希望发生的.我在 JUnit 中遗漏了什么吗?

In both cases above, the unit test runs which is what I'm hoping to have happen. Am I missing something in JUnit?

推荐答案

我在这里假设您希望在断言失败时测试通过，但如果断言成功，那么测试也应该通过.

I'm assuming here that you want the test to pass if your assert fails, but if the assert succeeds, then the test should pass as well.

>

最简单的方法是使用 测试规则.TestRule 提供了在运行测试方法之前和之后执行代码的机会.下面是一个例子:

The easiest way to do this is to use a TestRule. TestRule gives the opportunity to execute code before and after a test method is run. Here is an example:

public class ExpectedFailureTest {
    public class ExpectedFailure implements TestRule {
        public Statement apply(Statement base, Description description) {
            return statement(base, description);
        }

        private Statement statement(final Statement base, final Description description) {
            return new Statement() {
                @Override
                public void evaluate() throws Throwable {
                    try {
                        base.evaluate();
                    } catch (Throwable e) {
                        if (description.getAnnotation(Deprecated.class) != null) {
                            // you can do whatever you like here.
                            System.err.println("test failed, but that's ok:");
                        } else {
                            throw e;
                        }
                    }
                }
            };
        }
    }

    @Rule public ExpectedFailure expectedFailure = new ExpectedFailure();

    // actually fails, but we catch the exception and make the test pass.
    @Deprecated
    @Test public void testExpectedFailure() {
        Object o = null;
        o.equals("foo");
    }

    // fails
    @Test public void testExpectedFailure2() {
        Object o = null;
        o.equals("foo");
    }
}

首先，注意第一个方法被标记为@Deprecated.我使用它作为我想忽略任何断言失败的方法的标记.你可以随意识别方法，这只是一个例子.

First, note that the first method is marked as @Deprecated. I'm using this as a marker for the method for which I want to ignore any assertion failures. You can do whatever you like to identify the methods, this is just an example.

接下来，在 ExpectedFailure#apply() 中，当我执行 base.evaluate() 时，我会捕获任何 Throwable(包括 AssertionError)，并且如果该方法标有注释@Deprecated，我忽略错误.您可以根据版本号、某些文本等执行任何您喜欢的逻辑来决定是否应该忽略错误.您还可以将动态确定的标志传递给 ExpectedFailure 以允许它在某些版本号上失败:

Next, in the ExpectedFailure#apply(), when I do the base.evaluate(), I'm catching any Throwable (which includes AssertionError) and if the method is marked with the annotation @Deprecated, I ignore the error. You can perform whatever logic you like to decide whether you should ignore the error or not, based on version number, some text, etc. You can also pass a dynamically determined flag into ExpectedFailure to allow it to fail for certain version numbers:

public void unmarshalledDocumentHasExpectedValue() {
    doc = unmarshaller.unmarshal(getResourceAsStream("mydoc.xml"));

    expectedFailure.setExpectedFailure(doc.getVersionNumber() < 3000);

    final ST title = doc.getTitle();
    assertThat(doc.getTitle().toStringContent(), equalTo("Expected"));
}

有关更多示例，请参阅ExternalResource, 和 ExpectedException

For further examples, see ExternalResource, and ExpectedException

如果你想把你的测试标记为忽略而不是成功，它会变得有点复杂，因为测试在执行之前被忽略，所以你必须回顾性地将测试标记为忽略，这将涉及构建你自己的 Runner.首先，请参阅我对 的回答如何在套件中定义 JUnit 方法规则?.或者再问一个问题.

If you want to mark you tests as Ignored rather than Success, it becomes a bit more complex, because tests are ignored before they are executed, so you have to retrospectively mark a test as ignored, which would involve constructing your own Runner. To give you a start, see my answer to How to define JUnit method rule in a suite?. Or ask another question.

这篇关于将单元测试标记为 JUnit 中的预期失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！