Jenkins Build/Pipeline作业-作业在树中的列表/布局中的排序列表 [英] Jenkins Build / Pipeline job - Job's listing in tree / layout ordered listing
问题描述
对于给定的构建管道作业(其在构建中或在构建操作中具有"Trigger build on other projects"的下游作业)，是否有可能获得树状列表视图，该视图显示了调用哪个管道作业N的内容，子作业按调用顺序(顺序或并行)与该管道的子build#一起运行build#.
例如:如果我的管道作业具有以下视图: 然后，
我希望获得类似于(如果我只是以简单的文本格式)的排名最高的列表:
vac-3.0-src:52 called: vac-3.0-unit-test-main:37
vac-3.0-unit-test-main:37 called: vac-3.0-unit-testA:36
vac-3.0-unit-test-main:37 called: vac-3.0-unit-testB:36
vac-3.0-unit-test-main:37 called: vac-3.0-unit-testC:35
vac-3.0-unit-test-main:37 called: vac-3.0-unit-testD:35
vac-3.0-unit-test-main:37 called: vac-3.0-unit-testReporting:35
vac-3.0-unit-testReporting:35 called: vac-3.0-integration-test-main:28
vac-3.0-integration-test-main:28 called: vac-3.0-integration-testA:27
vac-3.0-integration-test-main:28 called: vac-3.0-integration-testB:27
vac-3.0-integration-testB:27 called: vac-3.0-acceptance-test:25
vac-3.0-acceptance-test:25 called: vac-3.0-configure-something:24
vac-3.0-configure-something:24 called: vac-3.0-perform-someaction:23
vac-3.0-perform-someaction:23 called: vac-3.0-preview-step:22
vac-3.0-preview-step:22 called: vac-3.0-deb-delivery-job:27
vac-3.0-preview-step:22 called: vac-3.0-rpm-el6:23
vac-3.0-preview-step:22 called: vac-3.0-vagrant-provision:20
vac-3.0-preview-step:22 called: vac-3.0-vagrant-run:21
vac-3.0-vagrant-run:21 called: vac-3.0-demo:10
或者可以以更健壮的结构方式来显示此信息，即它可以是JSON blob，其中父作业的结构将按照流水线运行/给定顺序包含其调用的所有作业(并行/顺序).
我使用Jenkins API(即/api/xml or /api/json?pretty=true&depth=10或更高版本)尝试了主作业的URL(通过curl)，但它没有提供我想要的信息(与给定的管道运行有关).
该信息在管道视图上可视化(根据图像)，并且有关子项目的某些信息在给定的Jenkins作业的仪表板(管道的一部分)上可用，但顺序不存在.
如果您尝试解决此问题并且有任何解决方案来获取此数据，我们将不胜感激.进行此工作的原因是针对给定的管道运行水平查找指标(而不是针对管道的每个单独工作垂直查找指标，因为我已经有了总时间，构建号，结果等的垂直/单独工作指标)，但是如何我可以根据给定的管道运行关联每个作业的指标，这就是我想要得到的.
如果上面的图像示例足够大，我们可以在此处引用较小的运行图像快照:
我看到了一个可能的解决方案，不确定是否有帮助，但肯定是一种尝试.
算法步骤:
==============
1)维护直接的父子文件(即JobA:JobB，JobA:JobC，JobA:JobC，JobC:JobD等)，即该文件将告诉每个Job X ，什么是直接的子/下游工作.通过Jenkins Groovy脚本，可以轻松生成/使用该脚本. PS:您可以在此文件中添加更多列，即JobA:JobB:Build:Sequential或JobA:JobB:Test:Parallel，以获取更好的水平指标来计算周转时间/每给定步骤(构建，测试，部署等)，以及父作业是按顺序还是与两个或多个作业并行地调用了子作业)，并相应地计算指标.
2)在构建管道视图"配置(设置)中，设置编号.的作业显示为1.
PS :如果要捕获该主管道作业的给定管道版本号，则可以将其设置为5、10或更多. /p>
出于测试目的，我仅显示1条管道构建运行.
3)在Linux中，使用curl，在构建管道视图的名称( PS )上获取查看源代码" HTML页面信息:是不在主要管道作业上).
即，在这种情况下，**不适用于jobA或xxvt-main或**，但请使用视图名称" URL(显示整个管道).假设视图名称(通过Build Pipeline View插件)创建为" MyPipelineView "
例如:curl -s http://my-jenkins-server:8080/view/MyPipelineView/ > /tmp/9.txt
这将为您提供HTML内容.
将此信息存储在某个文件中(临时).假设我将其存储在/tmp/9.txt
3)运行以下命令以获取作业的内部版本号.根据第二个较小的管道图像(在我的帖子中)，其输出将是:
grep -o "\"extId\":\"[a-zA-Z0-9_-][a-zA-Z0-9_-]*#[0-9][0-9]*\"" /tmp/9.txt
这将为您提供输出(使用sed/cut使其更整洁):
"extId":"xxvt_main#157"
"extId":"xxvt_splunk_run_collect_operation#29"
"extId":"xxvt_splunk_run_process_operation#29"
"extId":"xxvt_splunk_update_date_restart_splunk#29"
"extId":"xxvt_splunk_get_jenkins_data#38"
"extId":"xxvt_splunk_get_clearquest_dr_data#47"
4)现在，使用父子(直接关系)文件(在项目1中生成)，对于给定的管道运行，您具有上述输出，我们可以使用该文件来创建最终的构建管道树文件，即
xxvt_main#157 called: xxvt_splunk_get_jenkins_data#38
xxvt_main#157 called: xxvt_splunk_get_clearquest_dr_data#47
xxvt_main#157 called: xxvt_splunk_run_collect_operation#29
xxvt_splunk_run_collect_operation#29 called: xxvt_splunk_run_process_operation#29
xxvt_splunk_run_process_operation#29 called: xxvt_splunk_update_date_restart_splunk#29
5)知道与给定运行相关的 作业名称及其其构建号后，我们可以谨慎地使用Jenkins的api/json?pretty=true&depth=1 or 2 or 3，来获取我们想要获取指标的字段，并最终以自己喜欢的任何格式创建.csv文件/创建文件，其中将包含给定管道运行的指标-水平.
Is it possible that for a given Build Pipeline job (which has downstream jobs either in the build or post build action as "Trigger build on other projects"), I can get a Tree listing view showing which Pipeline job# N called, what child jobs in the calling order (sequential or parallel) with child build# for that pipeline run build#.
For ex: If my pipeline job has this view: then,
I'm expecting to get a listing of the top run similar to (in case I just put in simple text format):
vac-3.0-src:52 called: vac-3.0-unit-test-main:37
vac-3.0-unit-test-main:37 called: vac-3.0-unit-testA:36
vac-3.0-unit-test-main:37 called: vac-3.0-unit-testB:36
vac-3.0-unit-test-main:37 called: vac-3.0-unit-testC:35
vac-3.0-unit-test-main:37 called: vac-3.0-unit-testD:35
vac-3.0-unit-test-main:37 called: vac-3.0-unit-testReporting:35
vac-3.0-unit-testReporting:35 called: vac-3.0-integration-test-main:28
vac-3.0-integration-test-main:28 called: vac-3.0-integration-testA:27
vac-3.0-integration-test-main:28 called: vac-3.0-integration-testB:27
vac-3.0-integration-testB:27 called: vac-3.0-acceptance-test:25
vac-3.0-acceptance-test:25 called: vac-3.0-configure-something:24
vac-3.0-configure-something:24 called: vac-3.0-perform-someaction:23
vac-3.0-perform-someaction:23 called: vac-3.0-preview-step:22
vac-3.0-preview-step:22 called: vac-3.0-deb-delivery-job:27
vac-3.0-preview-step:22 called: vac-3.0-rpm-el6:23
vac-3.0-preview-step:22 called: vac-3.0-vagrant-provision:20
vac-3.0-preview-step:22 called: vac-3.0-vagrant-run:21
vac-3.0-vagrant-run:21 called: vac-3.0-demo:10
OR this information can be presented in a more robust structural manner i.e. it can be JSON blob where a parent job has structure which will have all jobs that it called (parallel/sequence) in the pipeline run / given order.
I tried the main job's URL (via curl) with Jenkins API i.e. /api/xml or /api/json?pretty=true&depth=10 or more but it doesn't give me the information that I'm looking for (related to a given pipeline run).
This information is visually available on the pipeline view (as per the image) and some information about subprojects is available on a given Jenkins job's dashboard (which was part of the pipeline) but the order is not there.
I'll appreciate if you have tried to solve this and have any solution to get this data. The reason for this effort is to find metrics horizontally for a given pipeline run (Rather than vertically for each individual job which are part of pipeline as I already have the vertical / individual job metrics for Total time, build#, result etc) but how I can relate each individual job's metrics for a given pipeline run, is what I'm trying to get.
If the above image example is big enough, we can refer a smaller run image snapshot here:
I see one possible solution, not sure if that's helpful but surely it's an attempt.
Algorithm Steps:
==============
1) Maintain a Direct Parent-Child file (i.e. JobA:JobB, JobA:JobC, JobA:JobC, JobC:JobD, ....) i.e. this file will tell that for each JobX, what's are the direct Sub-child/downstream jobs of that. Via Jenkins Groovy script, this can be easily generated/available. PS: You can add more columns to this file i.e. JobA:JobB:Build:Sequential or JobA:JobB:Test:Parallel to get even better horizontal metrics for calculating turnaround time / per given step (build, test, deploy, etc) and whether a parent job called the child job in a sequence or in parallel with two or more jobs) and calculate the metrics accordingly.
2) Inside "Build Pipeline View" Configure (Settings), set the no. of jobs to be displayed as 1.
PS: You can set this to whatever 5, 10, or more if you want to capture a given pipeline build# of that main pipeline job.
For testing purpose, I'm showing only 1 pipeline build run.
3) In Linux, use curl, get the "View Source" HTML page information on the build-pipeline-view's NAME (PS: This is NOT on the main pipeline job).
i.e. **not for jobA or xxvt-main or ** in this case, but use the View Name URL (which shows the whole pipeline). Let's assume the view name (via Build Pipeline View plugin) was created as "MyPipelineView"
ex: curl -s http://my-jenkins-server:8080/view/MyPipelineView/ > /tmp/9.txt
This will give you the HTML content.
Store this information in some file (Temporary). Let's assume I stored it in /tmp/9.txt
3) Run the following command to get the job's build#s. As per the second smaller pipeline image (in my post), the output of that will be:
grep -o "\"extId\":\"[a-zA-Z0-9_-][a-zA-Z0-9_-]*#[0-9][0-9]*\"" /tmp/9.txt
This will give you output like (use sed/cut to make it more cleaner):
"extId":"xxvt_main#157"
"extId":"xxvt_splunk_run_collect_operation#29"
"extId":"xxvt_splunk_run_process_operation#29"
"extId":"xxvt_splunk_update_date_restart_splunk#29"
"extId":"xxvt_splunk_get_jenkins_data#38"
"extId":"xxvt_splunk_get_clearquest_dr_data#47"
4) Now you have the above output for a given pipeline run, using the Parent-Child (direct relationship) file (which we generated in bullet 1), we can use that to create our final Build Pipeline Tree file i.e.
xxvt_main#157 called: xxvt_splunk_get_jenkins_data#38
xxvt_main#157 called: xxvt_splunk_get_clearquest_dr_data#47
xxvt_main#157 called: xxvt_splunk_run_collect_operation#29
xxvt_splunk_run_collect_operation#29 called: xxvt_splunk_run_process_operation#29
xxvt_splunk_run_process_operation#29 called: xxvt_splunk_update_date_restart_splunk#29
5) Upon knowing the a given run related job-name and its build#, we can use Jenkins's api/json?pretty=true&depth=1 or 2 or 3 carefully, to fetch fields we want to fetch for metrics and finally create/come up with a .csv file in whatever format you like, which will have metrics for a given pipeline run - HORIZONTALLY.
这篇关于Jenkins Build/Pipeline作业-作业在树中的列表/布局中的排序列表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
