在詹金斯表现上建立多个工作 [英] building multiple jobs in jenkins performance

问题描述

在Jenkins中，我有100个Java项目.每个都有自己的构建文件. 每次我想清除构建文件并再次编译所有源文件时. 使用bulkbuilder插件，我尝试编译所有作业.使100个作业并行运行. 但是性能很差.如果工作需要1分钟，则需要单独进行.批量处理需要20分钟.批处理大小越大，花费的时间越多..我在功能强大的服务器上运行此命令，因此不会出现内存和CPU问题.

In Jenkins I have 100 java projects. Each has its own build file. Every time I want clear the build file and compile all source files again. Using bulkbuilder plugin I tried compling all the jobs.. Having 100 jobs run parallel. But performance is very bad. Individually if the job takes 1 min. in the batch it takes 20mins. More the batch size more the time it takes.. I am running this on powerful server so no problem of memory and CPU.

请建议我如何克服这个问题.需要在詹金斯中进行哪些配置. 我正在使用war文件启动jenkins.

Please Suggest me how to over come this.. what configurations need to be done in jenkins. I am launching jenkins using war file.

谢谢..

推荐答案

即使您说您有足够的内存和CPU资源，但似乎意味着在增加并行运行作业的数量时会遇到某种瓶颈.我认为这是可以理解的.即使我不是Java开发人员，我也认为大多数Java构建工具都可以在内部并行化构建. IE.建立一个作业可能会消耗多个CPU内核和大量内存.

Even though you say you have enough memory and CPU resources, you seem to imply there is some kind of bottleneck when you increase the number of parallel running jobs. I think this is understandable. Even though I am not a java developer, I think most of the java build tools are able to parallelize build internally. I.e. building a single job may well consume more than one CPU core and quite a lot of memory.

因此，我建议您需要监视构建服务器并尝试使用不同的批处理大小来找到最佳数量.您应该执行例如构建正在运行时显示"vmstat 5"，看看您是否还有空闲的cpu.还要注意磁盘I/O.如果增加了批处理大小，但磁盘I/O却没有增加，则说明您正在消耗所有I/O容量，如果增加了批处理大小，这可能无济于事.

Because of this I suggest you need to monitor your build server and experiment with different batch sizes to find an optimal number. You should execute e.g. "vmstat 5" while builds are running and see if you have idle cpu left. Also keep an eye on the disk I/O. If you increase the batch size but disk I/O does not increase, you are consuming all of the I/O capacity and it probably will not help much if you increase the batch size.

找到最佳批处理大小(即为构建服务器配置多少个执行器)后，您可以调整其他内容以使处理速度更快:

When you have found the optimal batch size (i.e. how many executors to configure for the build server), you can maybe tweak other things to make things faster:

• 请尽量少花时间检查代码.配置SCM插件以删除不受版本控制的文件，而不是在构建开始之前删除工作区.如果使用git，则可以使用本地参考存储库，也可以进行浅表克隆或类似的操作.

• Try to spend as little time checking out code as possible. Instead of deleting workspace before build starts, configure the SCM plugin to remove files that are not under version control. If you use git, you can use a local reference repo or do a shallow clone or something like that.

您还可以尝试使用SSD磁盘来加快速度

You can also try to speed things up by using SSD disks

您可以获取更多服务器，在它们上运行Jenkins从服务器，并利用多个服务器的cpu和I/O容量，而不仅仅是一个服务器.

You can get more servers, run Jenkins slaves on them and utilize the cpu and I/O capacity of multiple servers instead of only one.

这篇关于在詹金斯表现上建立多个工作的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！