pytest测试失败时如何不将Jenkins作业标记为FAILURE [英] How to not mark Jenkins job as FAILURE when pytest tests fail

问题描述

我有一个带有管道的Jenkins设置，该管道使用pytest运行一些测试套件.有时测试失败，有时测试环境崩溃(随机HTTP超时，外部库错误等).作业会解析XML测试结果，但是只要pytest返回非零，构建就会被标记为FAILURE.

I have a Jenkins setup with a pipeline that uses pytest to run some test suites. Sometimes a test fails and sometimes the test environment crashes (random HTTP timeout, external lib error, etc.). The job parses the XML test result but the build is marked as FAILURE as long as pytest returns non-zero.

即使测试失败，我也希望詹金斯从pytest获取退出代码零，但我也希望将其他错误标记为失败.是否有任何pytest选项可以解决此问题?我发现pytest-custom_exit_code，但它只能抑制空测试套件错误.也许一些詹金斯选项或bash片段?

I want Jenkins to get exit code zero from pytest even if there are failed tests but I also want other errors to be marked as failures. Are there any pytest options that can fix this? I found pytest-custom_exit_code but it can only suppress the empty test suite error. Maybe some Jenkins option or bash snippet?

我的groovy管道的简化版本:

A simplified version of my groovy pipeline:

pipeline {
    stages {
        stage ('Building application') {
            steps {
                sh "./build.sh"
            }
        }
        stage ('Testing application') {
            steps {
                print('Running pytest')
                sh "cd tests && python -m pytest"
            }
            post {
                always {
                    archiveArtifacts artifacts: 'tests/output/'
                    junit 'tests/output/report.xml'
                }
            }
        }
    }
}

我尝试捕获退出代码1(表示某些测试失败 )，但Jenkins仍然收到退出代码1，并将该版本标记为FAILURE:

I have tried to catch exit code 1 (meaning some tests failed) but Jenkins still received exit code 1 and marks the build as FAILURE:

sh "cd tests && (python -m pytest; rc=\$?; if [ \$rc -eq 1 ]; then exit 0; else exit \$rc; fi)"

推荐答案

我的解决方案是在 pytest-custom-exit-code 并创建提取请求.

My solution was to implement the support myself in pytest-custom-exit-code and create a pull request.

从插件的0.3.0版本开始，我可以使用pytest --suppress-tests-failed-exit-code获得所需的行为.

From version 0.3.0 of the plugin I can use pytest --suppress-tests-failed-exit-code to get the desired behavior.

这篇关于pytest测试失败时如何不将Jenkins作业标记为FAILURE的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！