在管道中并行限制詹金斯 [英] throttling jenkins parallel in pipeline

问题描述

我在下面的代码中看到了这条消息在 JENKINS-44085.

如果我已经有一个包含 50 个项目的分支地图，但我想一次将它们并行 5 个，我需要如何修改此代码?我的代码已经在一个名为分支的 var 中有一个包含 50 个项目的映射.

//将一些项目放入队列以允许该数量的分支运行for (int i=0;i

解决方案

这个问题有点老了，但对我来说这个问题昨天也很相关.在某些情况下，您的 Jenkins 工作在 Jenkins 上可能很轻，但在其他系统上却很高，因此您希望针对该系统限制它.在我看来，每个构建代理使用 max executors 不是正确的方法，因为如果您的 Jenkins 集群可扩展，您将不得不调整内容.

要回答您的问题，您可能需要执行以下操作:

• 使用数字索引0"、1"等创建分支地图.
• 在您粘贴的代码的 try 块中，有如下内容:build(my_branches[name])

至少这就是我之前使用相同解决方法的方式.但是后来有人在工作中指出了一个更好的解决方案.我还在您提到的 Jira 票证中评论了这个更简单的解决方案.它需要可锁定资源插件:https://wiki.jenkins.io/display/JENKINS/可锁定+资源+插件

• 转到:http://<你的 Jenkins URL>/configure 并添加 X 带有标签XYZ"的可锁定资源.
• 在您的代码中这样使用:

<上一页>定义测试 = [:]为了 (...) {def test_num="$i"测试["$test_num"] = {锁定(标签:XYZ"，数量:1，变量:锁定"){println "锁定的资源:${env.LOCKED}"构建(作业:jobName，等待:true，参数:参数)}}}平行测试

这样做的好处是您可以在不同的工作中使用它.在我们的例子中，不同的作业在 XYZ 上都有负载，因此拥有这些全局锁非常方便.

I came across this message with the code below in JENKINS-44085.

If I already have a map of branches that contains 50 items, but I want to parallel them 5 at a time, how do I need to modify this code? My code already has a map of 50 items in a var named branches.

// put a number of items into the queue to allow that number of branches to run
for (int i=0;i<MAX_CONCURRENT;i++) {
    latch.offer("$i")
}

for (int i=0; i < 500; i++) {
    def name = "$i"
    branches[name] = {
        def thing = null
        // this will not allow proceeding until there is something in the queue.
        waitUntil {
            thing = latch.pollFirst();
            return thing != null;
        }
        try {
            echo "Hello from $name"
            sleep time: 5, unit: 'SECONDS'
            echo "Goodbye from $name"
        }
        finally {
           // put something back into the queue to allow others to proceed
            latch.offer(thing)
        }
    }
}

timestamps {
    parallel branches
}

解决方案

This question is a bit old, but for me the problem was also relevant yesterday. In some cases your Jenkins jobs may be light on Jenkins but high on some other system, so you want to limit it for that system. In my opinion using max executors per build agent is not the right way to do that because if your Jenkins cluster scales you will have to adjust stuff.

To answer your question, you probably want to do something like this:

• Create a branches map with numeric indexes "0", "1", etc.
• In the try block of that code you pasted have something like: build(my_branches[name])

At least that's how I was using that same workaround before. But then someone at work pointed out a better solution. I also commented this simpler solution in the Jira ticket you refered to. It requires the Lockable Resources Plugin: https://wiki.jenkins.io/display/JENKINS/Lockable+Resources+Plugin

• Go to: http://<your Jenkins URL>/configure and add X lockable resources with label "XYZ".
• Use in your code as such:

def tests = [:]
for (...) {
    def test_num="$i"
    tests["$test_num"] = {
        lock(label: "XYZ", quantity: 1, variable: "LOCKED") {
            println "Locked resource: ${env.LOCKED}"
            build(job: jobName, wait: true, parameters: parameters)
        }
    }
}
parallel tests 

The nice thing about this is that you can use this across different jobs. In our case different jobs have a load on XYZ so having these global locks are very handy.

这篇关于在管道中并行限制詹金斯的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！