如何跨阶段重用先前创建的工作区 [英] How to re-use previously created workspace across stages

问题描述

我面临一个问题，我在管道中定义了两个阶段，两个阶段都在同一节点上运行，并且需要在同一工作空间中运行.

I am facing an issue where I have two stages defined in my pipeline that both are run on the same node and need to be run in the same workspace.

这些阶段的第一阶段最初在我的主节点上运行，但是在定义的步骤快要结束时，必须将一些文件存储到另一个节点上.

The first of these stages runs on my master node initially, but towards the end of the steps defined has to unstash some files onto a different node.

然后，第二阶段只需要继续在我的主服务器上运行，并依赖于从第一阶段安装的某些模块.

The second stage then just needs to continue on my master, and relies on some modules that were installed from the first stage.

这是我的管道，可以更好地解释:

Here is my pipeline to better explain:

#!groovy
pipeline {
  agent { label 'master' }
  stages {
    stage('Build') { // 1. Running on master in /var/lib/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ
      steps {
        sh '''
          npm install
          bower install
          gulp set-staging-node-env
          gulp prepare-staging-files
          gulp webpack
        '''
        stash includes: 'dist/**/*', name: 'builtSources'
        stash includes: 'config/**/*', name: 'appConfig'
        node('Protractor') { // 2. Running on vnccentos7 in /var/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ
          dir('/opt/foo/deploy/') {
            unstash 'builtSources'
            unstash 'appConfig'
          }
        }        
      }
    }
    stage('Unit Tests') {
      agent { label 'master' } // 3. Running on master in /var/lib/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ@2
      steps {
        parallel (
          "Jasmine": {
            sh 'gulp karma-tests-ci'
          },
           "Mocha": {
            sh 'gulp mocha-tests'
          }
        )
      }
    }
  }
}

如您所见，我在每个阶段\节点的开头添加了注释，用于显示我看到的詹金斯输出，了解分配了哪些工作空间.

As you can see, I have added comments at the start of each stage\node used to show the jenkins output I see for what workspaces get allocated.

我面临的问题是单元测试阶段失败，因为它尝试使用一些找不到的节点模块.这些出现在创建的第一个工作空间中，这是我希望此阶段继续使用的位置，因此不要使用新的带后缀'@ 2'的工作空间.

The problem I am facing is that the Unit Tests stage fails as it tries to use some node modules it cannot find. These are present in the first workspace that gets created which is where I want this stage to continue using, so not using a new '@2' suffixed workspace.

是否可以告诉Jenkins在管道中保留先前创建的工作区?

Is there a way to tell the Jenkins to preserve previously created workspaces in the pipeline?

编辑

我正在猜测，因为我在下一阶段再次指定了代理{label:'master'}，这是什么情况下要创建新的工作区?我应该使用node方法吗?这样可以允许使用相同的工作空间吗?

I am guessing as I have specified agent {label:'master'} again in my next stage, this is what cases a new workspace to get created? Should I be using the node approach instead? Would that allow the same workspace to get used?

我实际上已经尝试在单元测试阶段的每个并行步骤中使用node('master'){...}，但是这些仍然使用@ 2后缀的工作区，而不是原始的工作区.

I actually have tried using node('master'){...} around each of the parallel steps in my Unit Tests stage, but these still use the @2 suffixed workspace rather than the original.

我看过其他线程在谈论如何避免重复使用同一工作空间，否则可能会遇到文件锁定问题.他们建议改为在步骤之间归档\取消归档工作空间.

I have seen other threads talking about how you should not re-use the same workspace as you could run into issues with file locks. They suggest instead to archive\unarchive the workspace between steps.

我还看到了一些方法，可以将工作空间路径存储在变量中，然后在以后使用它，这对我来说很合适，但是我还没有找到任何声明性的语法示例，只是古怪的.

I have also seen approaches where you can store the workspace path in a variable and use this later on which sounds good for my case, but I havent found any declarative syntax samples, only groovy ones.

编辑2

我现在已经尝试了几种方法，其中包括将分配的工作空间从第一阶段保存到变量中，并在以后的阶段中在ws(...)指令中使用:

I have now tried a few approaches involving saving the allocated workspace from the first stage into a variable and using in a ws(...) directive in the later stages:

pipeline {
  agent { label 'master' }
  stages {
    stage('Build') {
      steps {
        script {
          def workspace = pwd()
        }
        sh '''
          npm install
          bower install
          gulp set-staging-node-env
          gulp prepare-staging-files
          gulp webpack
        '''
        stash includes: 'dist/**/*', name: 'builtSources'
        stash includes: 'config/**/*', name: 'appConfig'
        node('Protractor') {
          dir('/opt/foo/deploy/') {
            unstash 'builtSources'
            unstash 'appConfig'
          }
        }        
      }
    }
    stage('Unit Tests') {
      steps {
        parallel (
          "Jasmine": {
            node('master') {
              ws("${workspace}"){
                sh 'gulp karma-tests-ci'
              }
            }
          },
          "Mocha": {
            node('master') {
              ws("${workspace}"){
                sh 'gulp mocha-tests'
              }
            }
          }
        )
      }
      post {
        success {
          sh 'gulp combine-coverage-reports'
          sh 'gulp clean-lcov'
          publishHTML(target: [
            allowMissing: false,
            alwaysLinkToLastBuild: false,
            keepAll: false,
            reportDir: 'test/coverage',
            reportFiles: 'index.html',
            reportName: 'Test Coverage Report'
          ])
        }
      }
    }
    }
}

我确实尝试只是从单元测试"阶段中删除了第二个代理声明，但是该阶段仍保留在我不想执行的量角器"节点上.因此，按照此处的答案\注释，然后在每个并行步骤中使用节点块，并使用ws块，如您所见，

I did try just removing the second agent declaration from the Unit Tests stage but the stage remained on my Protractor node which I didnt want it to do. So following the answers\comments here I then used node blocks around each of my parallel steps and used the ws blocks as you can see,

该阶段失败，我从日志中看到它没有使用从第一阶段分配的工作空间(不带@后缀):

The stage fails, and I can see from the logs that its not using the workspace allocated from the first stage (without the @ suffixes on):

[Jasmine] Running on master in /var/lib/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ@2
[Pipeline] [Jasmine] {
[Pipeline] [Jasmine] ws
[Jasmine] Running in /var/lib/jenkins/workspace/_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ@2@2
[Pipeline] [Jasmine] {
[Pipeline] [Jasmine] sh
[Jasmine] [_Pipelines_IACT-Jenkinsfile-UL3RGRZZQD3LOPY2FUEKN5XCY4ZZ6AGJVM24PLTO3OPL54KTJCEQ@2@2] Running shell script
[Jasmine] + gulp karma-tests-ci
[Jasmine] [08:27:01] No gulpfile found

它甚至带有@ 2的双后缀，所以我不确定它现在在做什么.

Its even double suffixing with @2, so I'm unsure what its doing now.

推荐答案

不确定是否适合您的用例，但是此示例脚本显示了如何在不同阶段之间共享相同的节点/工作区.容器:

Not sure if it fits your use case, but this example script shows how to share the same node/workspace between different stages & containers:

此外，如果您正在为特定阶段运行Docker代理，同时在顶层指定代理{标签'whatever'}，则可以确保此阶段将使用与其余管道相同的节点和工作空间:

Additionally, if you're running a Docker agent for a specific stage while specifying agent { label 'whatever' } at the top level, you can ensure that this stage will use the same node and workspace as the rest of the Pipeline:

pipeline {
  agent {
    label 'whatever'
  }
  stages {
    stage('build') {
      steps {
        sh "./build-artifact.sh"
      }
    }
    stage('test in docker') {
      agent {
        docker {
          image 'ubuntu:16.04'
          reuseNode true
        }
      }
      steps {
        sh "./run-tests-in-docker.sh"
      }
    }
  }
}

https://github.com/jenkinsci/pipeline-model-definition-plugin/wiki/Controlling-your-build-environment#reusing-nodeworkspace-with-per-stage-docker-agents

这篇关于如何跨阶段重用先前创建的工作区的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！