合并多个Jenkins作业的测试覆盖率和测试结果 [英] Merging test coverage and test results from multiple Jenkins jobs

问题描述

背景:

我们有一个用Java编写的相当大的REST API，正在结合单元测试和功能测试进行测试.在测试它时需要有许多变化，特别是在功能级别上.单元测试活在树中，而功能测试则位于单独的代码存储库中.

We have a rather large REST API written in Java that we're testing with combination of unit and functional tests. There are many variations that are required when testing it, particularly at the functional level. While the unit tests live in-tree, the functional tests are in a separate code repository.

我们目前正在使用Jacoco进行测试覆盖，并使用TestNG来运行单元测试，尽管我认为我的问题的答案应适用于其他工具组合.

We are currently using Jacoco for test coverage and TestNG for running our unittests, though I believe answers to my question should be applicable to other tool combinations.

我们在Jenkins中有几个不同的工作，这些工作是由对主要项目的签入触发的.这些任务包括运行诸如Coverity之类的工具的任务以及一些不同的功能测试任务.这些作业由初始提交触发，该提交不被视为绿色"，直到所有下游作业成功完成为止.

We have several different jobs in Jenkins that are triggered by a check-in to the primary project. These include jobs that run tools like Coverity as well as several different functional test jobs. These jobs are triggered by the initial commit, which is not considered to be "green", until all of the downstream jobs complete successfully.

问题:

我们如何获取覆盖率报告(如Jacoco二进制文件和TestNG xml文件)并将其合并以显示所有测试的总代码覆盖率?具体来说，如果它们存在于同一作业/目录中，我们知道如何组合它们，但是这些文件分布在可能在不同时间运行的多个Jenkins作业中.

How do we take coverage reports (like the Jacoco binaries and the TestNG xml files) and combine them to show total code coverage over all of our tests? Specifically, we know how to combine them if they are present in the same job/directory, but these files are spread across multiple Jenkins jobs which may be running at different times.

根据我的经验，最常见的处理方法是使用 升级版插件触发所有作业，然后在出现时拉出其工件它完成了触发工作.但是，当您要进行一两个以上的工作时，我觉得这种扩展性不太好.当您在主项目中可能有多个变体(旧版本等)时，尤其如此.

In my experience, the most commonly accepted way of handling this is to use the Promoted Builds Plugin to trigger all jobs, then pull their artifacts when it completes down to the triggering job. I don't feel like this scales very well, however, when you have more than one or two jobs that you're attempting to roll-up. This is especially true when you may have more than one variation on the master project (old releases, etc).

我知道可以在Jenkins中对文件进行指纹识别，从而使我们知道-.jar与Jobs A，B和C中使用的版本相同.是否存在可以根据该文件检索与模式匹配的所有文件的插件?是否存在其他指纹文件?

I understand it is possible to fingerprint files in Jenkins such that we know that -.jar is the same version used in Jobs A, B, and C. Does a plugin exist that can retrieve all files matching a pattern based on the existence of a different fingerprinted file?

另一种解决方案(可能会从ant/groovy脚本运行)是将测试数据推送到与git commit哈希绑定的目录中，并在基于汇总的作业中检索所有此类数据在基础项目的git commit哈希上.

One alternative solution (which would probably be run from an ant/groovy script), is to push test data to a directory somewhere that is tied to a git commit hash, and retrieve all such data in a roll-up job based on the git commit hash of the base project.

有没有简单的方法可以做到这一点?有没有人想出其他更好的方法来解决这个问题?

Are there any simple ways to do this? Has anyone figured out any better other ways to solve this problem?

谢谢

迈克尔

推荐答案

面对类似的问题，调整了jacoco maven插件配置以合并jacoco结果.基本上将jacoco-unit.exec和jacoco-it.exec合并为一个二进制文件，然后通过管道步骤将其合并到Jenkins上.

Faced similar issue, tweaked jacoco maven plugin config to merge jacoco results. Basically merged jacoco-unit.exec and jacoco-it.exec into one binary and published that merged result on Jenkins via pipeline step.

pom.xml:

            <plugin>
                    <inherited>false</inherited>
                    <groupId>org.jacoco</groupId>
                    <artifactId>jacoco-maven-plugin</artifactId>
                    <version>${jacoco.agent.version}</version>
                    <executions>

                        <execution>
                            <id>merge-results</id>
                            <phase>post-integration-test</phase>
                            <goals>
                                <goal>merge</goal>
                            </goals>
                            <configuration>
                                <fileSets>
                                    <fileSet>
                                        <directory>${project.parent.build.directory}</directory>
                                        <includes>
                                            <include>jacoco-*.exec</include>
                                        </includes>
                                    </fileSet>
                                </fileSets>
                                <destFile>${project.parent.build.directory}/jacoco.exec</destFile>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>

Jenkinsfile:

Jenkinsfile:

            echo 'Publish Jacoco trend'
            step($class: 'JacocoPublisher',
                    execPattern: '**/jacoco.exec',
                    classPattern: '**/classes',
                    sourcePattern: '**/src/main/java',
            )

但是，您仍然必须通过另一个构建步骤从多个Jenkins构建中获取jacoco二进制文件，或显式指定其位置.

However you still have to fetch jacoco binaries from several Jenkins builds by another build step or specify their locations explicitly.

这篇关于合并多个Jenkins作业的测试覆盖率和测试结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！