脚本化 jenkinsfile 并行阶段 [英] Scripted jenkinsfile parallel stage

问题描述

我正在尝试使用 groovy DSL 编写脚本化的 Jenkinsfile，它将在一组阶段中具有并行步骤.

I am attempting to write a scripted Jenkinsfile using the groovy DSL which will have parallel steps within a set of stages.

这是我的詹金斯文件:

node {   
stage('Build') {
    sh 'echo "Build stage"'
}

stage('API Integration Tests') {
    parallel Database1APIIntegrationTest: {
        try {
            sh 'echo "Build Database1APIIntegrationTest parallel stage"'
        }
        finally {
            sh 'echo "Finished this stage"'
        }               

    }, Database2APIIntegrationTest: {
        try {
            sh 'echo "Build Database2APIIntegrationTest parallel stage"'
        }
        finally {
            sh 'echo "Finished this stage"'
        }

    }, Database3APIIntegrationTest: {
        try {
            sh 'echo "Build Database3APIIntegrationTest parallel stage"'
        }
        finally {
            sh 'echo "Finished this stage"'
        }
    }
}

stage('System Tests') {
    parallel Database1APIIntegrationTest: {
        try {
            sh 'echo "Build Database1APIIntegrationTest parallel stage"'
        }
        finally {
            sh 'echo "Finished this stage"'
        }               

    }, Database2APIIntegrationTest: {
        try {
            sh 'echo "Build Database2APIIntegrationTest parallel stage"'
        }
        finally {
            sh 'echo "Finished this stage"'
        }

    }, Database3APIIntegrationTest: {
        try {
            sh 'echo "Build Database3APIIntegrationTest parallel stage"'
        }
        finally {
            sh 'echo "Finished this stage"'
        }
    }
}
}

我希望有 3 个阶段:构建；集成测试和系统测试.在这两个测试阶段，我希望并行执行 3 组测试，每组测试针对不同的数据库.

I want to have 3 stages: Build; Integration Tests and System Tests. Within the two test stages, I want to have 3 sets of the tests executed in parallel, each one against a different database.

我有 3 个可用的执行程序.一个在 master 上，2 个 agent 上，我希望每个并行步骤都能在任何可用的执行器上运行.

I have 3 available executors. One on the master, and 2 agents and I want each parallel step to run on any available executor.

我注意到，在运行我的管道后，我只能看到 3 个阶段，每个阶段都标记为绿色.我不想查看该阶段的日志来确定该阶段中的任何并行步骤是否成功/不稳定/失败.

What I've noticed is that after running my pipeline, I only see the 3 stages, each marked out as green. I don't want to have to view the logs for that stage to determine whether any of the parallel steps within that stage were successful/unstable/failed.

我希望看到测试阶段中的 3 个步骤 - 标记为绿色、黄色或红色(成功、不稳定或失败).

I want to be seeing the 3 steps within my test stages - marked as either green, yellow or red (Success, unstable or failed).

我已经考虑将测试扩展到他们自己的阶段，但已经意识到不支持并行阶段(有谁知道这是否会被支持?)，所以我不能这样做，因为管道会花费太多时间很长时间才能完成.

I've considered expanding the tests out into their own stages, but have realised that parallel stages are not supported (Does anyone know whether this will ever be supported?), so I cannot do this as the pipeline would take far too long to complete.

任何见解将不胜感激，谢谢

Any insight would be much appreciated, thanks

推荐答案

在 Jenkins 脚本管道中，parallel(...) 需要一个 Map 来描述要构建的每个阶段.因此，您可以以编程方式预先构建构建阶段，这是一种允许灵活的串行/并行切换的模式.
我使用了与此类似的代码，其中 prepareBuildStages 返回一个 Map 列表，每个 List 元素按顺序执行，而 Map 描述了此时的并行阶段.

In Jenkins scripted pipeline, parallel(...) takes a Map describing each stage to be built. Therefore you can programatically construct your build stages up-front, a pattern which allows flexible serial/parallel switching.
I've used code similar to this where the prepareBuildStages returns a List of Maps, each List element is executed in sequence whilst the Map describes the parallel stages at that point.

// main script block
// could use eg. params.parallel build parameter to choose parallel/serial 
def runParallel = true
def buildStages

node('master') {
  stage('Initialise') {
    // Set up List<Map<String,Closure>> describing the builds
    buildStages = prepareBuildStages()
    println("Initialised pipeline.")
  }

  for (builds in buildStages) {
    if (runParallel) {
      parallel(builds)
    } else {
      // run serially (nb. Map is unordered! )
      for (build in builds.values()) {
        build.call()
      }
    }
  }

  stage('Finish') {
      println('Build complete.')
  }
}

// Create List of build stages to suit
def prepareBuildStages() {
  def buildStagesList = []

  for (i=1; i<5; i++) {
    def buildParallelMap = [:]
    for (name in [ 'one', 'two', 'three' ] ) {
      def n = "${name} ${i}"
      buildParallelMap.put(n, prepareOneBuildStage(n))
    }
    buildStagesList.add(buildParallelMap)
  }
  return buildStagesList
}

def prepareOneBuildStage(String name) {
  return {
    stage("Build stage:${name}") {
      println("Building ${name}")
      sh(script:'sleep 5', returnStatus:true)
    }
  }
}

生成的管道显示为:

并行块内可以嵌套的内容有一定的限制，参考管道文档以获取确切的详细信息.不幸的是，大部分参考文献似乎偏向于声明式管道，尽管它不如脚本式(恕我直言)灵活.管道示例 页面最有帮助.

There are certain restrictions on what can be nested within a parallel block, refer to the pipeline documentation for exact details. Unfortunately much of the reference seems biased towards declarative pipeline, despite it being rather less flexible than scripted (IMHO). The pipeline examples page was the most helpful.

这篇关于脚本化 jenkinsfile 并行阶段的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！