在 testng 中重试失败的测试用例 [英] Retry failed test case in testng

问题描述

我正在做一个 testng 项目，我的目标是让 testng 自动重试失败的测试用例.例如，第一轮有10个测试用例，5个失败.所以在第一轮之后，我让 testng 选择了 5 个失败的测试用例并重新运行它们.第二轮，可能有2个失败的测试用例，然后我重新运行这2个.

I am working on a testng project and my goal is that let testng auto retry the failed test cases. For example, there are 10 test cases in the first round and 5 failed. So after first round, I let testng select the 5 failed test cases and rerun them again. In the second round, maybe there are 2 failed test cases, then I rerun this 2 agian.

我尝试过 IRetryAnalyzer，但它是不同的.IRetryAnalyzer 会立即重试失败的测试用例，而不是每轮结束.

I have tried the IRetryAnalyzer but it is different. The IRetryAnalyzer is retrying the failed test cases immediatelly instead of the end of each round.

所以目前我想在 ISuiteListener 中使用 onStart 和 onFinish 调用重试.在这种情况下，我实现了这样的 onFinish 方法:

So currently I want to call the retry using onStart and onFinish in the ISuiteListener. In this case I implement the onFinish method like this:

@Override
public void onFinish(ISuite suite) {
    logger.info("Round " + retryCounter
            + " Testing suit stops. onFinish method is invoked.");
    if (!doRetry()) {
        logger.info("Retry finished.");
        cleanReport(suite);
        return;
    }

    // I want to remove the passed cases here
    // and create a suite to run the failed test cases.
    suite.run();
}

那有可能吗?或者对此要求有更好的想法.

So is it possible do that? Or any better idea for this requirement.

推荐答案

看起来这个问题很老了，但仍然没有答案，所以这是我的解决方案.

It looks like the question is old, but still unanswered, so here is my solution.

首先，你必须像这样实现 IRetryAnalyzer 接口:

Firstly, you have to implement IRetryAnalyzer interface like this:

    public class Retry implements IRetryAnalyzer {

    private int retryCount = 0;
    private int maxRetryCount = 1;

    public boolean retry(ITestResult result) {
        if (retryCount < maxRetryCount) {
            System.out.println("Retrying test " + result.getName() + " with status "
                    + getResultStatusName(result.getStatus()) + " for the " + (retryCount+1) + " time(s).");
            retryCount++;
            return true;
        }
        return false;
    }

    public String getResultStatusName(int status) {
        String resultName = null;
        if(status==1)
            resultName = "SUCCESS";
        if(status==2)
            resultName = "FAILURE";
        if(status==3)
            resultName = "SKIP";
        return resultName;
    }
}

这里，maxRetryCount 是您希望重新运行失败测试的次数.测试失败后立即重新运行.

Here, maxRetryCount is the number of times, you want your failed tests to be re-run. Tests are re-run immediately after their fail.

其次，实现 IAnnotationTransformer 接口:

Secondly, implement IAnnotationTransformer interface:

public class RetryListener implements IAnnotationTransformer {

    @Override
    public void transform(ITestAnnotation testannotation, Class testClass,
                          Constructor testConstructor, Method testMethod)   {
        IRetryAnalyzer retry = testannotation.getRetryAnalyzer();

        if (retry == null)  {
            testannotation.setRetryAnalyzer(Retry.class);
        }

    }
}

现在，在带有测试的 TestNG xml 套件中，您应该添加侦听器:

Now, in your TestNG xml suite with tests, you should add listener:

<listeners>
    <listener class-name="Utils.reRunTest.RetryListener"/>
</listeners>

您也可以将其添加到您的 pom.xml 文件中:

You can also add it to your pom.xml file:

 <properties>
     <property>
         <name>usedefaultlisteners</name>
         <value>false</value>
     </property>
     <property>
         <name>listener</name>
         <value>org.testng.reporters.EmailableReporter2, org.testng.reporters.XMLReporter, org.testng.reporters.FailedReporter, Utils.reRunTest.RetryListener
         </value>
     </property>

您可能还希望重新运行测试不影响测试总数.这是通过实现 TestListener 接口并将其添加为与 RetryAnalyzer 一起的侦听器来完成的.从测试总数中删除不稳定测试和失败测试的实现是:

You may also want your test re-runs not to affect total number of tests. This is done, by implementing TestListener interface and adding it as a listener alongside with RetryAnalyzer. The implementation that will remove flaky tests and failed tests from total number of tests is:

public class TestListener implements ITestListener {
    @Override
    public void onFinish(ITestContext context) {
        Set<ITestResult> failedTests = context.getFailedTests().getAllResults();
        Set<ITestResult> skippedTests = context.getSkippedTests().getAllResults();
        for (ITestResult temp : failedTests) {
            ITestNGMethod method = temp.getMethod();
            if (context.getFailedTests().getResults(method).size() > 1) {
                failedTests.remove(temp);
            } else {
                if (context.getPassedTests().getResults(method).size() > 0) {
                    failedTests.remove(temp);
                }
            }
        }
        for (ITestResult temp : skippedTests) {
            ITestNGMethod method = temp.getMethod();
            if (context.getSkippedTests().getResults(method).size() > 1) {
                skippedTests.remove(temp);
            } else {
                if (context.getPassedTests().getResults(method).size() > 0) {
                    skippedTests.remove(temp);
                }
            }
        }
    }

    public void onTestStart(ITestResult result) {
    }

    public void onTestSuccess(ITestResult result) {
    }

    public void onTestFailure(ITestResult result) {
    }

    public void onTestSkipped(ITestResult result) {
    }

    public void onTestFailedButWithinSuccessPercentage(ITestResult result) {
    }

    public void onStart(ITestContext context) {
    }
}

希望有所帮助.P.S 不要在 TestNG 6.9.10 版中使用它`因为它有一些问题，你重新运行测试总是会通过的，不管它们的真实状态.

Hope that helps. P.S Don't use it with TestNG version 6.9.10 `cause it has some issues and you re-run tests will always pass, disregarding their real state.

这篇关于在 testng 中重试失败的测试用例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！