Jenkins-如何找出哪些测试反复失败? [英] Jenkins - How to find out which tests failed repeatedly?

问题描述

我是CI&的新手.詹金斯.我有一个Java项目，该项目运行基于Testng的自动化测试.这些测试在詹金斯(Jenkins)中作为工作定期进行.有时，该工作长时间反复失败.但是，在每次运行中，测试失败的次数是不同的.我想看看哪些测试反复失败，比如说最后5次运行.这样，我可以减少必须调查的测试失败的次数.在过去5次运行中未能完全重现的故障可以忽略或稍后进行调查.

I am new to CI & Jenkins. I have a Java project which runs Testng based automated tests. The tests run regularly as a job in Jenkins. Sometimes, the job fails repeatedly for a long time. But, in each run, the number of test failures is different. I want to see which tests failed repeatedly in, say the last 5 runs. With that, I can reduce the number of test failures which I have to investigate. The failures which do not repeat all in the last 5 runs could be ignored or investigated later.

有人可以建议我如何找出在连续运行中哪些测试反复失败吗?

Can someone please suggest how I can find out which tests failed repeatedly in some consecutive runs ?

PS -这是一个示例，解释为什么我希望在过去的几次运行中看到测试失败.

PS - Here is an example to explain why I want to see test failures in the past few runs.

Run 1 - test1 failed, test3 failed, test10 failed.
Run 2 - test3 failed, test17 failed.
Run 3 - test1 failed, test3 failed.

查看了这3次运行后，我们发现test3在所有运行中均失败.但是，其他测试至少通过了三个运行之一.我知道在我开始这3次运行之前，test3曾经通过.这告诉我test3可能由于系统中的错误而失败，但是其他测试可能由于系统中的间歇性问题而失败.此信息使我可以决定首先调查哪个测试，即test3.如有需要，我可以在以后查看其他测试.

After looking at these 3 runs, we see that test3 fails in all the runs. But, the other tests pass in at least one of the three runs. I know that test3 used to pass before I started these 3 runs. This tells me that test3 might be failing due to a bug in the system, but the other tests might be failing due to intermittent issues in the system. This info allows me to decide which test to investigate first, i.e. test3. I can look at the other tests later if needed.

推荐答案

您可以从管道中的构建对象获取测试失败的次数:您可以制作一个小报告，并将其附加到构建中，如下所示:

You can take the number of times a test failed from the build's object in a pipeline: you can make a small report and attach it to your build like so:

@NonCPS
def getRepeatedlyFailingTests(int timesFailedAtLeast = 2) {
  currentBuild.rawBuild.getAction(hudson.tasks.junit.TestResultAction.class)
    .getFailedTests()
    // Keep only tests that failed at least twice
    .findAll { it.age >= timesFailedAtLeast }
    .collect { [ "${it.className}.${it.name}".trim(), it.age ] }
}

def saveRepeatedlyFailingTestsReport() {
  def header = [ "test", "times-failed" ]
  def records = getRepeatedlyFailingTests()
  def report = "repeatedly-failing-tests.csv"
  writeCSV file: report, records: [ header ] + records, format: CSVFormat.EXCEL
  archiveArtifacts report
}

收集测试结果后致电saveRepeatedlyFailingTestsReport，您将在最新的构建工件链接中看到该报告.

Call saveRepeatedlyFailingTestsReport after you gather your test results and you'll see the report in your latest build artifacts link.

这篇关于Jenkins-如何找出哪些测试反复失败?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！