如何跨阶段重用以前创建的工作区 [英] How to re-use previously created workspace across stages

问题描述

我面临一个问题，我在管道中定义了两个阶段，这两个阶段都在同一个节点上运行，并且需要在同一个工作区中运行.

I am facing an issue where I have two stages defined in my pipeline that both are run on the same node and need to be run in the same workspace.

这些阶段中的第一个最初在我的主节点上运行，但在定义的步骤结束时必须将一些文件解散到不同的节点上.

The first of these stages runs on my master node initially, but towards the end of the steps defined has to unstash some files onto a different node.

然后第二阶段只需要在我的主人上继续，并依赖于从第一阶段安装的一些模块.

The second stage then just needs to continue on my master, and relies on some modules that were installed from the first stage.

这是我的管道，以便更好地解释:

Here is my pipeline to better explain:

#!groovy
pipeline {
  agent { label 'master' }
  stages {
    stage('Build') { // 1. Running on master in /var/lib/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ
      steps {
        sh '''
          npm install
          bower install
          gulp set-staging-node-env
          gulp prepare-staging-files
          gulp webpack
        '''
        stash includes: 'dist/**/*', name: 'builtSources'
        stash includes: 'config/**/*', name: 'appConfig'
        node('Protractor') { // 2. Running on vnccentos7 in /var/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ
          dir('/opt/foo/deploy/') {
            unstash 'builtSources'
            unstash 'appConfig'
          }
        }        
      }
    }
    stage('Unit Tests') {
      agent { label 'master' } // 3. Running on master in /var/lib/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ@2
      steps {
        parallel (
          "Jasmine": {
            sh 'gulp karma-tests-ci'
          },
           "Mocha": {
            sh 'gulp mocha-tests'
          }
        )
      }
    }
  }
}

如您所见，我在每个阶段节点的开头添加了注释，用于显示我看到的 jenkins 输出，以了解分配了哪些工作区.

As you can see, I have added comments at the start of each stage ode used to show the jenkins output I see for what workspaces get allocated.

我面临的问题是单元测试阶段失败，因为它试图使用一些它找不到的节点模块.这些存在于创建的第一个工作区中，我希望此阶段继续使用该工作区，因此不使用新的@2"后缀工作区.

The problem I am facing is that the Unit Tests stage fails as it tries to use some node modules it cannot find. These are present in the first workspace that gets created which is where I want this stage to continue using, so not using a new '@2' suffixed workspace.

有没有办法告诉 Jenkins 在管道中保留以前创建的工作区?

Is there a way to tell the Jenkins to preserve previously created workspaces in the pipeline?

编辑

我猜因为我在下一阶段再次指定了代理 {label:'master'}，这是什么情况下要创建新的工作区?我应该改用 node 方法吗?这会允许使用相同的工作区吗?

I am guessing as I have specified agent {label:'master'} again in my next stage, this is what cases a new workspace to get created? Should I be using the node approach instead? Would that allow the same workspace to get used?

我实际上已经尝试在我的单元测试阶段的每个并行步骤周围使用 node('master'){...}，但这些仍然使用@2 后缀的工作区而不是原创.

I actually have tried using node('master'){...} around each of the parallel steps in my Unit Tests stage, but these still use the @2 suffixed workspace rather than the original.

我已经看到其他线程在谈论如何不重复使用相同的工作区，因为您可能会遇到文件锁定问题.他们建议在步骤之间归档取消归档工作区.

I have seen other threads talking about how you should not re-use the same workspace as you could run into issues with file locks. They suggest instead to archiveunarchive the workspace between steps.

我还看到了一些方法，您可以将工作区路径存储在一个变量中，稍后再使用它，这对我的情况来说听起来不错，但我还没有找到任何声明性语法示例，只有常规示例.

I have also seen approaches where you can store the workspace path in a variable and use this later on which sounds good for my case, but I havent found any declarative syntax samples, only groovy ones.

编辑 2

我现在尝试了几种方法，包括将第一阶段分配的工作区保存到一个变量中，并在后期阶段使用 ws(...) 指令:

I have now tried a few approaches involving saving the allocated workspace from the first stage into a variable and using in a ws(...) directive in the later stages:

pipeline {
  agent { label 'master' }
  stages {
    stage('Build') {
      steps {
        script {
          def workspace = pwd()
        }
        sh '''
          npm install
          bower install
          gulp set-staging-node-env
          gulp prepare-staging-files
          gulp webpack
        '''
        stash includes: 'dist/**/*', name: 'builtSources'
        stash includes: 'config/**/*', name: 'appConfig'
        node('Protractor') {
          dir('/opt/foo/deploy/') {
            unstash 'builtSources'
            unstash 'appConfig'
          }
        }        
      }
    }
    stage('Unit Tests') {
      steps {
        parallel (
          "Jasmine": {
            node('master') {
              ws("${workspace}"){
                sh 'gulp karma-tests-ci'
              }
            }
          },
          "Mocha": {
            node('master') {
              ws("${workspace}"){
                sh 'gulp mocha-tests'
              }
            }
          }
        )
      }
      post {
        success {
          sh 'gulp combine-coverage-reports'
          sh 'gulp clean-lcov'
          publishHTML(target: [
            allowMissing: false,
            alwaysLinkToLastBuild: false,
            keepAll: false,
            reportDir: 'test/coverage',
            reportFiles: 'index.html',
            reportName: 'Test Coverage Report'
          ])
        }
      }
    }
    }
}

我确实尝试从单元测试阶段删除第二个代理声明，但该阶段仍保留在我的量角器节点上，我不希望它这样做.因此，按照这里的答案评论，然后我在每个并行步骤周围使用节点块，并使用 ws 块，如您所见，

I did try just removing the second agent declaration from the Unit Tests stage but the stage remained on my Protractor node which I didnt want it to do. So following the answerscomments here I then used node blocks around each of my parallel steps and used the ws blocks as you can see,

阶段失败，我可以从日志中看到它没有使用从第一阶段分配的工作区(没有@后缀):

The stage fails, and I can see from the logs that its not using the workspace allocated from the first stage (without the @ suffixes on):

[Jasmine] Running on master in /var/lib/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ@2
[Pipeline] [Jasmine] {
[Pipeline] [Jasmine] ws
[Jasmine] Running in /var/lib/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ@2@2
[Pipeline] [Jasmine] {
[Pipeline] [Jasmine] sh
[Jasmine] [_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ@2@2] Running shell script
[Jasmine] + gulp karma-tests-ci
[Jasmine] [08:27:01] No gulpfile found

它甚至带有@2 的双重后缀，所以我不确定它现在在做什么.

Its even double suffixing with @2, so I'm unsure what its doing now.

推荐答案

不确定它是否适合您的用例，但此示例脚本展示了如何在不同阶段之间共享相同的节点/工作区 &容器:

Not sure if it fits your use case, but this example script shows how to share the same node/workspace between different stages & containers:

此外，如果您正在为特定阶段运行 Docker 代理，同时在顶层指定 agent { label 'whatever' }，您可以确保该阶段将使用与管道其余部分相同的节点和工作区:

Additionally, if you're running a Docker agent for a specific stage while specifying agent { label 'whatever' } at the top level, you can ensure that this stage will use the same node and workspace as the rest of the Pipeline:

pipeline {
  agent {
    label 'whatever'
  }
  stages {
    stage('build') {
      steps {
        sh "./build-artifact.sh"
      }
    }
    stage('test in docker') {
      agent {
        docker {
          image 'ubuntu:16.04'
          reuseNode true
        }
      }
      steps {
        sh "./run-tests-in-docker.sh"
      }
    }
  }
}

https://github.com/jenkinsci/pipeline-model-definition-plugin/wiki/Controlling-your-build-environment#reusing-nodeworkspace-with-per-stage-docker-agents

这篇关于如何跨阶段重用以前创建的工作区的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！