在管道中并行节流詹金斯 [英] throttling jenkins parallel in pipeline

问题描述

我通过以下代码看到了这条消息在 JENKINS-44085 中.

如果我已经有一个包含 50 个项目的分支地图，但我想一次将它们并行 5 个，我需要如何修改此代码?我的代码已经在一个名为分支的 var 中有一个包含 50 个项目的映射.

//将一些项目放入队列以允许运行该数量的分支for (int i=0;i

解决方案

这个问题有点老了，但对我来说，昨天的问题也很重要.在某些情况下，您的 Jenkins 工作在 Jenkins 上可能很少，但在其他系统上可能很高，因此您希望限制该系统的工作.在我看来，为每个构建代理使用最大执行器并不是正确的方法，因为如果您的 Jenkins 集群扩展，您将不得不调整内容.

要回答您的问题，您可能想要执行以下操作:

• 创建一个带有数字索引0"、1"等的分支映射.
• 在您粘贴的代码的 try 块中，有如下内容:build(my_branches[name])

至少我以前是这样使用相同的解决方法的.但是后来有人在工作中指出了一个更好的解决方案.我还在您提到的 Jira 票证中评论了这个更简单的解决方案.它需要可锁定资源插件:https://wiki.jenkins.io/display/JENKINS/Lockable+Resources+Plugin

• 转到:http:///configure 并添加标签为XYZ"的 X 可锁定资源.
• 在您的代码中使用:

<前>def 测试 = [:]为了 (...) {def test_num="$i"测试[$test_num"] = {锁(标签:XYZ"，数量:1，变量:锁定"){println "锁定资源:${env.LOCKED}"构建(作业:作业名称，等待:真，参数:参数)}}}平行测试

这样做的好处是您可以在不同的工作中使用它.在我们的例子中，不同的作业在 XYZ 上都有负载，因此拥有这些全局锁非常方便.

I came across this message with the code below in JENKINS-44085.

If I already have a map of branches that contains 50 items, but I want to parallel them 5 at a time, how do I need to modify this code? My code already has a map of 50 items in a var named branches.

// put a number of items into the queue to allow that number of branches to run
for (int i=0;i<MAX_CONCURRENT;i++) {
    latch.offer("$i")
}

for (int i=0; i < 500; i++) {
    def name = "$i"
    branches[name] = {
        def thing = null
        // this will not allow proceeding until there is something in the queue.
        waitUntil {
            thing = latch.pollFirst();
            return thing != null;
        }
        try {
            echo "Hello from $name"
            sleep time: 5, unit: 'SECONDS'
            echo "Goodbye from $name"
        }
        finally {
           // put something back into the queue to allow others to proceed
            latch.offer(thing)
        }
    }
}

timestamps {
    parallel branches
}

解决方案

This question is a bit old, but for me the problem was also relevant yesterday. In some cases your Jenkins jobs may be light on Jenkins but high on some other system, so you want to limit it for that system. In my opinion using max executors per build agent is not the right way to do that because if your Jenkins cluster scales you will have to adjust stuff.

To answer your question, you probably want to do something like this:

• Create a branches map with numeric indexes "0", "1", etc.
• In the try block of that code you pasted have something like: build(my_branches[name])

At least that's how I was using that same workaround before. But then someone at work pointed out a better solution. I also commented this simpler solution in the Jira ticket you refered to. It requires the Lockable Resources Plugin: https://wiki.jenkins.io/display/JENKINS/Lockable+Resources+Plugin

• Go to: http://<your Jenkins URL>/configure and add X lockable resources with label "XYZ".
• Use in your code as such:

def tests = [:]
for (...) {
    def test_num="$i"
    tests["$test_num"] = {
        lock(label: "XYZ", quantity: 1, variable: "LOCKED") {
            println "Locked resource: ${env.LOCKED}"
            build(job: jobName, wait: true, parameters: parameters)
        }
    }
}
parallel tests 

The nice thing about this is that you can use this across different jobs. In our case different jobs have a load on XYZ so having these global locks are very handy.

这篇关于在管道中并行节流詹金斯的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！