Jenkins 构建一个由许多 Maven 项目组成的产品?(使用 Jenkins Pipeline 插件?) [英] Jenkins building a product consisting of many Maven projects? (with Jenkins Pipeline plugin?)

问题描述

我们的产品由许多相互依赖的 Maven 项目组成.所有这些 Maven 项目都集中在一个项目中，该项目提供最终产品.

Maven 项目共享相同的生命周期.换句话说，它们不是由具有显式 <dependency> 更改以获取其他项目的较新版本的单独团队管理.相反，当有人在其中一个项目中更改某些内容时，结果应该直接进入最终产品，无需额外更改.

我们使用 Jenkins 作为我们的持续集成工具.

我们的主要愿望如下:

• 无需将所有项目间依赖项复制到 Jenkins 配置中:它们应该在一个地方，最好是 pom.xml 文件.
• 避免不必要的构建:在 SCM 更改时，仅构建可能受到影响的项目.
• 在钻石依赖的情况下(C 依赖于 B1 和 B2，它们都依赖于 A)，如果最低的 (A) 发生变化，那么最终产品 (C) 应该始终使用也用于构建/测试 B1 和 B2 的 A 版本.

问题:使用 Jenkins 进行此操作的最佳方法是什么?

我们目前正在考虑使用 Jenkins Pipeline 插件使用单个作业，该插件分析 Maven 依赖项和 SCM 更改，决定需要构建的内容和顺序，然后实际构建项目.

解决方案

为了满足您的要求，您可以依赖 maven 多模块构建的许多默认功能以及下面解释的附加配置.

从你的问题:

<块引用>

无需将所有项目间依赖项复制到 Jenkins 配置中:它们应该在一个地方，最好是 pom.xml 文件.

使用 aggregator/multi-module pom 你可以有一个入口点(pom.xml 文件)用于 maven 构建，然后构建所有已定义的模块:

<块引用>

具有模块的项目称为多模块或聚合器项目.模块是这个 POM 列出的项目，并作为一个组执行.一个 pom 打包的项目可以通过将一组项目作为模块列出来聚合它们的构建，这些模块是这些项目的相对目录.

即pom.xml文件如下:

<代码><项目><modelVersion>4.0.0</modelVersion><groupId>com.sample</groupId><artifactId>项目</artifactId><版本>0.0.1-SNAPSHOT</版本><包装>pom</包装><模块><module>module-1</module><module>module-2</module><module>module-n</module></模块></项目>

是一个最小的例子:定义将从这个 maven 项目开始构建的模块列表(其他 maven 项目)(一个空项目，其唯一目的是构建其他 maven 项目).注意 pom 包装 需要有模块，它告诉 maven这个项目只提供了一个 pom(没有更多的工件).

因此，您可以拥有一个根 Maven 聚合器项目，它将其他 maven 项目定义为其模块，并让一个 Jenkins 作业构建此入口点.

<小时>

另外，根据你的问题:

<块引用>

避免不必要的构建:在 SCM 更改时，仅构建可能受到影响的项目.

要满足此要求，您可以使用 incremental-build-插件:

<块引用>

因为它寻找对一个独特项目的模块的修改，所以 Maven 增量插件只对多模块项目敏感.如果检测到对模块的修改，则删除输出目录.

此插件将验证 pom 文件、资源、源、测试源、测试资源是否会在模块中发生变化，以及是否删除其输出目录.因此，节省了相关模块的构建时间以及整个多模块项目的链式构建时间(在这种情况下是我们自己的构建).

要启用此机制，您可以在上面的聚合器 pom 中进行配置，如下所示:

<代码><项目><modelVersion>4.0.0</modelVersion><groupId>com.sample</groupId><artifactId>项目</artifactId><版本>0.0.1-SNAPSHOT</版本><包装>pom</包装><模块><module>module-1</module><module>module-2</module><module>module-n</module></模块><属性><skip.incremental>true</skip.incremental></属性><构建><插件><插件><groupId>net.java.maven-incremental-build</groupId><artifactId>增量构建插件</artifactId><版本>1.6</版本><执行><执行><目标><目标>增量构建</目标></目标><配置><noIncrementalBuild>${skip.incremental}</noIncrementalBuild></配置></执行></执行></插件></插件></构建></项目>

在上面，我们简单地将 incremental-build-plugin 添加到聚合器构建和一个属性 skip.incremental，它将首先跳过插件执行构建一个空的聚合器，同时将其启用到模块中，如下所示:

<代码><项目><modelVersion>4.0.0</modelVersion><父母><groupId>com.sample</groupId><artifactId>项目</artifactId><版本>0.0.1-SNAPSHOT</版本></父母><artifactId>简单模块</artifactId><属性><skip.incremental>false</skip.incremental></属性></项目>

注意:在示例模块的上述 pom 中，我们将聚合器 pom 文件作为父级，因此使用 maven 继承 结合聚合(经典用法)，因此具有多模块构建以及由公共父级为所有声明的模块提供的公共构建治理(在这种情况下，公共构建治理提供了额外的 incremental-build-plugin 配置).此外，每个模块都重新配置 skip.incremental 属性以不跳过上述插件.这是一个技巧:现在插件只会在模块中执行，而不是在其根目录中(这没有意义，在这种情况下实际上会抛出错误).

显然，在相关的 Jenkins 作业中，在其 Source Code Management 部分中，我们不必在每次构建时配置全新的签出，否则将无法检测到任何更改(并且每次它都会从零开始，这实际上是发布版本的好习惯).

此外，您的 maven 命令不应调用 clean 生命周期，这也会在每次构建时删除 target，因此也无法检测到任何更改.

<小时>

此外，根据您的问题:

<块引用>

在钻石依赖"的情况下(C 依赖于 B1 和 B2，两者都依赖于 A)，如果最低的 (A) 发生变化，那么最终产品 (C) 应该始终使用 A 的版本还用于构建/测试 B1 和 B2.

Maven 将在多模块构建及其 reactor 机制:

<块引用>

Maven 中处理多模块项目的机制称为反应器.这部分 Maven 核心执行以下操作:

• 收集所有可用的模块来构建
• 按照正确的构建顺序对项目进行排序
• 按顺序构建选定的项目

默认情况下，reactor 也会因此创建一个构建顺序，并且总是在 在其依赖/消费者模块之前构建一个模块.这也意味着最后构建的模块实际上将是负责构建最终工件的模块，可交付产品取决于部分或所有其他模块(例如，负责交付 war 的模块文件可能是在多模块 webapp 项目中构建的最后一个文件).

<小时>

有关 Maven 的进一步相关阅读以及在某些内容未更改时跳过操作:

• maven-jar-plugin 提供forceCreation默认情况下已启用<块引用>

  要求 jar 插件构建一个新的 JAR，即使内容似乎都没有改变.默认情况下，此插件会查看输出 jar 是否存在并且输入是否未更改.如果这些条件为真，插件将跳过 jar 的创建.

• maven-compiler-plugin提供 useIncrementalCompilation，虽然目前无法正常工作.
• maven-war-plugin 提供recompressZippedFiles也可以用来加速构建和避免重新做某事的选项(在这种情况下切换到 false ):<块引用>

  表示是否应该再次压缩被添加到战争中的 zip 档案(jar、zip 等).再次压缩会导致压缩文件的大小变小，但会显着延长执行时间.
  默认:true

<小时>

更新

<块引用>

Maven 项目共享相同的生命周期.

此要求还将强制使用聚合器/多模块项目，但也可能适用于通过依赖项链接的不同项目.

<块引用>

它们不是由单独的团队管理，这些团队有明确的更改来获取其他项目的更新版本.

这一点也强制使用多模块项目.在多模块项目中，模块之间可能有不同的版本，但是通常的做法和准则是共享相同的版本，由父项目(聚合器)定义并在其模块之间级联.因此，其集中化和治理将避免错位和错误.

<块引用>

当有人在其中一个项目中更改某些内容时，结果应直接进入最终产品，无需额外更改.

同样，这将由多模块项目自动处理.每个使用 SNAPSHOT 版本，则无需更改其消费者项目(负责构建最终产品的项目)中的依赖项版本.然而，虽然 SNAPSHOT 版本在开发过程中确实很有帮助(并且被推荐)，但在交付最终产品时绝对不应该使用它们，因为构建可重现性将处于危险之中(也就是说，您以后可能无法重新构建相同的版本on，因为它依赖于 SNAPSHOT 版本，因此不是冻结版本).因此，SNAPSHOT 不是灵丹妙药，只能在产品 SDLC 的某些阶段使用，而不是作为最终确定和冻结的解决方案.

<小时>

更新 2
值得一看的是新的 Maven 增量模块构建器Maven 3.3.1+ 和 Java 7.

更多详情:

• 它的 Github 项目
• 其作者博文(Apache 的主要成员之一)Maven 贡献者)

We have a product that consists of many Maven projects that depend on each other. All of these Maven projects come together in a single project which delivers the end product.

The Maven projects share the same life cycle. In other words, they are not managed by separate teams of people with explicit <dependency> changes to pick up newer versions of other projects. Rather, when someone changes something in one of the projects, then the result should go directly into the end product without additional changes.

We use Jenkins as our Continuous Integration tool.

The main wishes we have are as follows:

• No need to copy all the inter-project dependencies to Jenkins configuration: these should be in a single place, ideally the pom.xml files.
• Avoid unnecessary builds: on an SCM change, only build the projects that are potentially affected.
• In case of diamond dependencies (C depends on both B1 and B2, which both depend on A), if the lowest (A) is changed, then the end product (C) should always use the version of A that was also used to build/test B1 and B2.

Question: What is the best approach to do this with Jenkins?

We are currently thinking to use a single job using the Jenkins Pipeline plugin, which analyzes the Maven dependencies and the SCM changes, decides what needs to be built and in which order, and then actually build the projects.

解决方案

In order to achieve your requirements you could rely on many of the default features of a maven multi-module build and an additional configuration explained below.

From your question:

No need to copy all the inter-project dependencies to Jenkins configuration: these should be in a single place, ideally the pom.xml files.

Using an aggregator/multi-module pom you can have a single entry point (a pom.xml file) for a maven build which would then build all of the defined modules:

A project with modules is known as a multimodule, or aggregator project. Modules are projects that this POM lists, and are executed as a group. An pom packaged project may aggregate the build of a set of projects by listing them as modules, which are relative directories to those projects.

That is, a pom.xml file like the following:

<project>
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.sample</groupId>
    <artifactId>project</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>pom</packaging>

    <modules>
        <module>module-1</module>
        <module>module-2</module>
        <module>module-n</module>
    </modules>
</project>

Is a minimal example: define a list of modules (other maven projects) which will be built starting from this maven project (an empty project which only purpose is to build other maven projects). Note the pom packaging required to have modules, it's telling maven that this project only provides a pom (no further artifacts).

You could hence have a root Maven aggregator project which would define the other maven projects as its modules and have a single Jenkins job building this entry point.

---

Additionally, from your question:

Avoid unnecessary builds: on an SCM change, only build the projects that are potentially affected.

To meet this requirement, you could use the incremental-build-plugin:

Because it looks for modification on modules of an unique project, the Maven Incremental Plugin takes it sens only on multi modules projects. If modification on a module is detected, the output directory is removed.

This plugin will verify whether any of the pom file, resources, sources, test sources, test resources would change in a module and if the case remove its output directory. As such, saving build time for a concerned module and in chain for the whole multi-module project (our own build, in this case).

To enable this mechanism, you can configure in the aggregator pom above, the following:

<project>
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.sample</groupId>
    <artifactId>project</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>pom</packaging>
    <modules>
        <module>module-1</module>
        <module>module-2</module>
        <module>module-n</module>
    </modules>

    <properties>
        <skip.incremental>true</skip.incremental>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>net.java.maven-incremental-build</groupId>
                <artifactId>incremental-build-plugin</artifactId>
                <version>1.6</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>incremental-build</goal>
                        </goals>
                        <configuration>
                            <noIncrementalBuild>${skip.incremental}</noIncrementalBuild>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

Above, we have simply added the incremental-build-plugin to the aggregator build and a property, skip.incremental, which will skip the plugin execution for the first build, the empty aggregator one, while enabling it into the modules as following:

<project>
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>com.sample</groupId>
        <artifactId>project</artifactId>
        <version>0.0.1-SNAPSHOT</version>
    </parent>
    <artifactId>simple-module</artifactId>

    <properties>
        <skip.incremental>false</skip.incremental>
    </properties>
</project>

Note: in the pom above of a sample module we are pointing at the aggregator pom file as a parent, hence using maven inheritance in combination with aggregation (a classic usage) and as such having a multi-module build plus a common build governance provided by the common parent for all the declared modules (in this case, the common build governance provides the additional incremental-build-plugin configuration). Moreover, each module re-configures the skip.incremental property to not skip the aforementioned plugin. That's a trick: now the plugin will only be executed in modules, not in its root (which would not make sense and in this case would actually throw an error otherwise).

Obviously, in the related Jenkins job, in its Source Code Management section we don't have to configure a fresh new check-out as part of each build, otherwise no changes would be detectable (and everytime it would start from zero, which is a good practice for release builds actually).

Moreover, your maven command should not invoke the clean lifecycle, which would also remove the target at each build and as such make no change detectable either.

---

Furthermore, from your question:

In case of 'diamond dependencies' (C depends on both B1 and B2, which both depend on A), if the lowest (A) is changed, then the end product (C) should always use the version of A that was also used to build/test B1 and B2.

Maven will take care of this requirement as part of a multi-module build and its reactor mechanism:

The mechanism in Maven that handles multi-module projects is referred to as the reactor. This part of the Maven core does the following:

• Collects all the available modules to build
• Sorts the projects into the correct build order
• Builds the selected projects in order

By default, the reactor will also hence create a build order and always build a module before its dependent/consumer module. That also means that the last built module will actually be the module responsible of building the final artifact, the deliverable product depending on part or all of the other modules (for instance, a module responsible of delivering a war file will probably be the last one to build in a multi-module webapp project).

---

Further related reading concerning Maven and skip actions when something is unchanged:

• The maven-jar-plugin provides the forceCreation which by default is already enabled

  Require the jar plugin to build a new JAR even if none of the contents appear to have changed. By default, this plugin looks to see if the output jar exists and inputs have not changed. If these conditions are true, the plugin skips creation of the jar.

• The maven-compiler-plugin provides the useIncrementalCompilation, although not properly working at the moment.
• The maven-war-plugin provides the recompressZippedFiles option which could also be used to speed up builds and avoid re-doing something (to switch to false in this case):

  Indicates if zip archives (jar,zip etc) being added to the war should be compressed again. Compressing again can result in smaller archive size, but gives noticeably longer execution time.
  Default: true

---

Update

The Maven projects share the same life cycle.

This requirement would also enforce the usage of an aggregator/multi-module project, however may also apply to different projects linked through dependencies indeed.

they are not managed by separate teams of people with explicit changes to pick up newer versions of other projects.

This point also enforces the usage of a multi-module project. In a multi-module project you may have different versions among modules, however the common practice and guideline is to share the same version, defined by the parent project (the aggregator) and cascaded across its modules. As such, its centralization and governance will avoid misalignments and mistakes.

when someone changes something in one of the projects, then the result should go directly into the end product without additional changes.

Again, this would be handled automatically by a multi-module project. The same would happen with different projects each using SNAPSHOT versions, then no need to change the dependency version in its consumer project (the one responsible of building the final product). However, while SNAPSHOT versions are really helpful (and recommended) during development, they should definitely not be used when delivering the final product since build reproducibility would be in danger (that is, you may not be able to re-build the same version later on since it was relying on SNAPSHOT versions, hence not frozen versions). Hence, SNAPSHOT is not a silver bullet solution and should be used only during certain phases of the SDLC of the product, not as a finalized and frozen solution.

---

Update 2
Worth to also look at the new Maven Incremental Module Builder for Maven 3.3.1+ and Java 7.

More details on:

• Its Github project
• Its author blog post (one the main Apache Maven contributors)

这篇关于Jenkins 构建一个由许多 Maven 项目组成的产品?(使用 Jenkins Pipeline 插件?)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！