在 Maven 中使用缓存的工件来避免冗余构建? [英] Using cached artifacts in Maven to avoid redundant builds?

问题描述

我有一个存储在 Git 中的 Maven 3 多模块项目(~50 个模块).多名开发人员正在处理和构建此代码，我们还有自动构建机器，可以在每次推送时运行冷构建.

I have a Maven 3 multi-module project (~50 modules) which is stored in Git. Multiple developers are working on this code and building it, and we also have automated build machines that run cold builds on every push.

大多数单独的更改日志会更改相当少的模块中的代码，因此每次更改都重新构建整个源代码树是浪费时间.但是，我仍然希望运行父项目构建的最终结果与构建整个代码库的结果相同.而且我不想手动开始对模块进行版本控制，因为这会成为交叉版本更新的噩梦.

Most individual changelogs alter code in a fairly small number of modules, so it's a waste of time to rebuild the entire source tree with every change. However, I still want the final result of running the parent project build to be the same as if it had built the entire codebase. And I don't want to start manually versioning modules, as this would become a nightmare of criss-crossing version updates.

我想做的是添加一个插件来拦截构建或安装中的某个步骤，并获取模块内容的哈希值(最好从 Git 中提取)，然后在共享的二进制存储库中查找存储在其下的工件哈希.如果找到，它会使用该工件，甚至不执行完整构建.如果它在缓存中没有找到任何东西，它会照常执行构建，然后将其工件存储在缓存中.重建任何具有依赖项(直接或瞬态)且本身有缓存未命中的模块也很好.

What I would like to do is add a plugin which intercepts some step in build or install, and takes a hash of the module contents (ideally pulled from Git), then looks in a shared binary repository for an artifact stored under that hash. If one is found, it uses that artifact and doesn't even execute the full build. If it finds nothing in the cache it performs the build as normal, then stores its artifact in the cache. It would also be good to rebuild any modules which have dependencies (direct or transient) which themselves had a cache miss.

有没有什么东西已经做了这样的事情?如果没有，将它添加到 Maven 的最干净的方法是什么?看起来插件可能能够完成它，但是对于几个部分，我无法找到连接到 Maven 的正确方法.具体:

Is there anything out there which does anything like this already? If not, what would be the cleanest way to go about adding it to Maven? It seems like plugins might be able to accomplish it, but for a couple pieces I'm having trouble finding the right way to attach to Maven. Specifically:

1. 如何拦截安装"目标以检查缓存，并且仅在缓存未命中时调用模块的本机"安装目标?
2. 插件应该如何将状态从一个模块传递到另一个模块，以了解发生了哪些缓存未命中，以便强制重新构建具有更改的依赖项?

我也愿意采用完全不同的方式来实现相同的最终结果(更少的冗余构建)，尽管解决方案越激进，在短期内对我的价值就越小.

I'm also open to completely different ways to achieve the same end result (fewer redundant builds) although the more drastic the solution the less value it has for me in the near term.

推荐答案

我之前实现了一个更复杂的解决方案，其中包含工件版本操作和部署到私有 Maven 存储库.但是，我认为这会更好地满足您的需求并且更简单:

I have previously implemented a more complicated solution with artifact version manipulation and deployment to private Maven repository. However, I think this will fit your needs better and is somewhat more simple:

1. 将您的构建拆分为多个构建(例如，每个模块使用 maven -pl 参数进行单个构建).
2. 在这些构建之间设置父子关系.(Bamboo 甚至对确定 Maven 依赖项有额外的支持，但我不确定它是如何工作的.)
3. 配置 Maven settings.xml 以使用不同的本地存储库位置 - 在构建工作目录中指定一个新目录.请参阅文档:https://maven.apache.org/guides/mini/guide-configuring-maven.html
4. 使用 mvn install 目标确保将新构建的工件添加到本地存储库
5. 使用 Bamboo 工件共享从本地存储库公开构建的工件 - 您可能应该过滤它以仅包含您感兴趣的包
6. 设置依赖构建以从父构建下载所有工件并将它们放入本地存储库的适当子目录(自定义为在工作目录中)

1. Split your build into multiple builds (e.g., with a single build per module using maven -pl argument).
2. Setup parent-child relationships between these builds. (Bamboo even has additional support for figuring out Maven dependencies, but I'm not sure how it works.)
3. Configure Maven settings.xml to use a different local repository location - specify a new directory inside your build working directory. See docs: https://maven.apache.org/guides/mini/guide-configuring-maven.html
4. Use mvn install goal to ensure newly built artifacts are added to local repository
5. Use Bamboo artifact sharing to expose built artifacts from local repository - you should probably filter this to include only the package(s) you're interested in
6. Set dependent builds to download all artifacts from parent builds and put them into proper subdirectory of local repository (which is customized to be in working directory)

由于 Bamboo 处理分支构建的父子关系的方式，这甚至应该适用于功能分支构建.

This should even work for feature branch builds thanks to the way Bamboo handles parent-child relations for branch builds.

请注意，这意味着 Maven 将重新下载所有其他依赖项，因此您应该使用本地网络上的代理私有 Maven 存储库，例如 Artifactory 或 Nexus.

Note that this implies that Maven will redownload all other dependencies, so you should use a proxy private Maven repository on local network, such as Artifactory or Nexus.

如果您愿意，我还可以描述我已经实现的更复杂的场景，包括修改工件版本和部署到私有 Maven 存储库.

If you want, I can also describe the more complicated scenario I've already implemented that involves modifying artifact versions and deploying to private Maven repository.

这篇关于在 Maven 中使用缓存的工件来避免冗余构建?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！