Jenkins在不同的代理上并行构建 [英] Jenkins parallel build on different agents

问题描述

我有一个Jenkins声明性管道的小例子，该管道应同时在"Windows"和"Linux"代理上运行.目标是动态构建配置矩阵(例如，发布/调试配置；不同的CMake参数；等等)，并使所有组合并行运行.但是，我陷入了建立从预备变量执行并行步骤的流水线的过程.

这是脚本的一个版本，其中在parallel{}块内显式指定了并行阶段:

pipeline {
    agent any
    stages {
        stage ("Parallel Build") {
            parallel {
                stage ("Windows") {
                    steps {
                        echo "TEST Windows"
                    }
                }
                stage ("Linux") {
                    steps {
                        echo "TEST Linux"
                    }
                }
            }
        }
    }
}

我的计划是在parallel{}块中动态创建阶段(取决于所需的配置)，但是我不确定语法或是否完全可能.

类似这样的东西:

def stage_list = {
    stage ("Windows") {          <=== How to correctly create the stage_list?
        steps {
            echo "TEST Windows"
        }
    }
    stage ("Linux") {
        steps {
            echo "TEST Linux"
        }
    }
}

pipeline { 
    agent any
    stages {    
        stage ("Parallel Build") {
                parallel stage_list <== How to replace a block with variable?
        }
    }
}

以上内容将在詹金斯中返回错误:

WorkflowScript: 17: Expected a block for parallel @ line 17, column 9.
           stage ("Parallel Build") {
           ^

WorkflowScript: 17: Expected one of "steps", "stages", or "parallel" for stage "Parallel Build" @ line 17, column 9.
           stage ("Parallel Build") {
           ^

有人知道怎么做吗?

---

编辑:在前两个答案之后，我想稍作更新.

我测试了创建stage_list变量的建议方法.但是，如果将parallel stage_list的调用放入原始结构中，则会得到与以前相同的错误.与script这样的

一起使用时，它可以很好地运行

pipeline { 
    agent any
    stages {    
        stage ("Parallel Build") {
            steps {
                script {
                    parallel stepsForParallel
                }
            }            
        }
    }
}

有人可以向我解释差异吗?为什么steps和script不能使用它们?

---

出于文档方面的原因，我想用解决问题的方法来结束问题:

在 Docker的管道语法中，

SmartToms答案和Jenkins的官方文档很明确声明性管道和脚本管道是需要分开处理的两种独立方法(请注意每个示例下面的切换脚本管道"链接).

解决我的问题的一种方法是使用脚本化管道-如果有人对此示例感兴趣，请使用解决方案

来自此

您可以在此处中找到有关并行Jenkins声明式管道的更多信息. >.

---

修改:

为什么它不能与steps和script一起使用，但不能没有它们?

根据此文档，我认为使用parallel带有列表的是 old 方法(声明式管道1.2之前的方法)，该方法要求在声明式管道中使用脚本化管道.

似乎新方法parallel(来自Declarative Pipeline 1.2)不能与列表一起使用.因此，要执行此操作，必须使用 old 脚本化管道方法，因此，您需要将命令pipeline stage_list与script封装在一起，而命令本身必须与steps封装.

您可以在此处处找到有关脚本管道和声明性管道的更多信息. /p>

I have a small example of a Jenkins declarative pipeline that should run on both 'Windows' and 'Linux' agents in parallel. The goal is to dynamically build a configuration matrix (e.g. Release / Debug configurations; different CMake parameters; etc.) and let all combinations run in parallel. However, I'm stuck in building a pipeline that executes the parallel steps from a prepared variable.

This is a version of the script where the parallel stages are specified explicitly within the parallel{} block:

pipeline {
    agent any
    stages {
        stage ("Parallel Build") {
            parallel {
                stage ("Windows") {
                    steps {
                        echo "TEST Windows"
                    }
                }
                stage ("Linux") {
                    steps {
                        echo "TEST Linux"
                    }
                }
            }
        }
    }
}

My plan is to create the stages in the parallel{} block dynamically (depending on the required configurations), but I'm not sure about the syntax or if this is possible at all.

Something like this:

def stage_list = {
    stage ("Windows") {          <=== How to correctly create the stage_list?
        steps {
            echo "TEST Windows"
        }
    }
    stage ("Linux") {
        steps {
            echo "TEST Linux"
        }
    }
}

pipeline { 
    agent any
    stages {    
        stage ("Parallel Build") {
                parallel stage_list <== How to replace a block with variable?
        }
    }
}

The above will return an error in Jenkins:

WorkflowScript: 17: Expected a block for parallel @ line 17, column 9.
           stage ("Parallel Build") {
           ^

WorkflowScript: 17: Expected one of "steps", "stages", or "parallel" for stage "Parallel Build" @ line 17, column 9.
           stage ("Parallel Build") {
           ^

Does anyone have an idea how this could be done?

---

Edit: After the first two answers I would like to update my question a bit.

I tested the suggested way of creating the stage_list variable. However, if I put this call of parallel stage_list into my original structure I get the same error as before. It runs well when using with script like this

pipeline { 
    agent any
    stages {    
        stage ("Parallel Build") {
            steps {
                script {
                    parallel stepsForParallel
                }
            }            
        }
    }
}

Can somebody explain the difference to me? Why does does it work with steps and script but not without them?

---

Edit 2: For documentation reasons I wanted to finish my question with what I did to solve the problem:

SmartToms answer and Jenkins' official documentation on Pipeline Syntax with Docker made clear that declarative pipeline and scripted pipeline are two separate approaches that need to be handled differently (pay attention to the "Toggle Scripted Pipeline" link below each example).

One way to solve my question is to use scripted pipeline - if anyone is interested in an example for this, here is a link to a gist with the pipeline script that shows the principle.

解决方案

From this documentation, this could be done like that :

// Creation of the stage_list
def stage_list = ["Windows", "Linux"]

// Creation of a map of stages
def stepsForParallel = stage_list.collectEntries {
    ["echoing ${it}" : transformIntoStage(it)]
}

// Run the stages in parallel
parallel stepsForParallel

// Creation of the stage
def transformIntoStage(inputString) {
    return {
        stage (inputString) {
            steps {
                echo "TEST "+inputString
            }
        }
    }
}

You can find more information about parallel Jenkins declarative pipeline here.

---

Edit:

Why does does it work with steps and script but not without them?

According to this documentation, I think that using parallel with a list is the old method (prior Declarative Pipeline 1.2), which required to use Scripted Pipeline within a Declarative Pipeline.

It seems that the new method parallel (from Declarative Pipeline 1.2) can not be used with a list. So, to do it you have to use the old Scripted Pipeline method, therefore you need to encapsulate your command pipeline stage_list with script which needs to be itself encapsulated with steps.

You can find more information about Scripted Pipeline and Declarative Pipeline here.

这篇关于Jenkins在不同的代理上并行构建的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！