用Maven处理依赖地狱的系统方法 [英] Systematic approach with Maven to deal with dependency hell

问题描述

我正在努力解决如何处理jar依赖地狱的问题。我有一个使用一些aws sdk的Maven-IntelliJ Scala项目。最近添加kinesis sdk引入了不兼容的Jackson版本。

I'm struggling with how to approach jar dependency hell. I have a Maven-IntelliJ Scala project that uses some aws sdk's. Recently adding the kinesis sdk has introduced incompatible versions of Jackson.

我的问题是：我如何系统地解决Jar地狱的问题？

My question is: how do I systemically approach the problem of Jar hell?

我理解类加载器以及maven在重复的Jars之间选择的方式，但我仍然对解决问题的实际步骤感到茫然。

I understand class loaders and how maven chooses between duplicate Jars, but I am still at a loss regarding actual practical steps to fix the issue.

我目前的尝试是基于反复试验，我在杰克逊的例子中概述了这里：

My attempts at the moment are based on trial and error, and I am outlining the here with the Jackson example:

• 首先，我看到了Jackson数据绑定ObjectMapper类的实际异常，在本例中是NoSuchMethodError。然后，我查看Jackson文档，了解添加或删除方法的时间。这通常非常繁琐，因为我手动检查每个版本的api文档（问题1：有更好的方法吗？）。


• 然后，我使用 mvn依赖：树找出我实际使用的Jackson版本（问题2：是否有一种自动方式询问maven哪个版本的jar是在使用中，而不是梳理树输出？）。


• 最后，我比较 mvn依赖：树在添加Kinesis SDK之前输出，之后，检测 mvn依赖关系：树输出中的差异，并希望看看Jackson版本是否发生了变化。 （问题3：当发生依赖性解析时，maven如何使用着色jar中的库？与其他任何一个相同？）。

• First, I see what the actual exception is, in this case NoSuchMethodError, on the Jackson data bindings ObjectMapper class. I then look at the Jackson docs to see when the method was added or removed. This is usually quite tedious, as I manually check the api docs for each version (question 1: is there a better way?).
• Then, I use mvn dependency:tree to figure out which version of the Jackson I am actually using (question 2: is there an automatic way of asking maven which version of a jar is in use, rather than combing through the tree output?).
• Finally, I compare the mvn dependency:tree output before adding the Kinesis SDK, and after, to detect differences in the mvn dependency:tree output, and hopefully see if the Jackson version changed. (question 3: How does maven use the libraries in shaded jars, when dependency resolution occurs? Same as any other?).

最后，在比较树输出后，我尝试在POM中明确添加最新的Jackson工作版本，以触发优先级maven依赖解析链。如果最新版本不起作用，我会添加下一个最新的库，依此类推。

Finally, after comparing the tree outputs, I try to add the lastest working version of Jackson explicitly in the POM, to trigger precedence in the maven dependency resolution chain. If the latest does not work, I add the next most recent lib, and so forth.

整个过程非常繁琐。除了我提出的具体问题，我也很好奇其他人对这个问题的系统方法。有没有人有他们使用的资源？

This entire procedure is incredibly tedious. Besides the specific questions I asked, I am also curious about other people's systemic approaches to this problem. Does any one have any resources that they use?

推荐答案

然后我查看Jackson文档，了解添加或删除方法的时间。这通常非常繁琐，因为我手动检查每个版本的api文档（问题1：有更好的方法吗？）

I then look at the Jackson docs to see when the method was added or removed. This is usually quite tedious, as I manually check the api docs for each version (question 1: is there a better way?)

为了检查API（破坏）兼容性，有几种工具可以自动分析罐子并为您提供正确的信息。从这个 Stack Overflow帖子中可以找到一些方便工具的好提示。

JAPICC 似乎相当不错。

To check API (breaking) compatibility there are several tools which would automatically analyze jars and provide you the right information. From this Stack Overflow post there are nice hints for some handy tools.
JAPICC seems quite good.

然后，我使用 mvn依赖：tree 来确定我实际使用的Jackson版本（问题2：是否有自动版）询问maven正在使用哪个版本的jar而不是梳理树输出的方法？）

Then, I use mvn dependency:tree to figure out which version of the Jackson I am actually using (question 2: is there an automatic way of asking maven which version of a jar is in use, rather than combing through the tree output?)

maven-dependency-tree 肯定是要走的路，但你可以从范围开始过滤掉，只能得到你实际想要的东西，使用它的 包含 选项为以下：

The maven-dependency-tree is definitely the way to go, but you can filter out since the beginning the scope and only get what you are actually looking for, using its includes option as following:

mvn dependency:tree -Dincludes=<groupId>

注意：您还可以向包含选项，格式为 groupId：artifactId：type：version 或使用通配符，如 *：artifactId 。

note: you can also provide further info to the includes option in the form groupId:artifactId:type:version or use wildcards like *:artifactId.

这似乎是一个小小的提示，但在具有许多依赖关系的大型项目中缩小其输出量是非常有帮助的。通常，只需 groupId 就足够作为过滤器， *：artifactId 可能是最快的，但如果你是寻找特定的依赖。

It seems a small hint, but in large projects with many dependencies narrowing down its output is of great help. Normally, simply the groupId should be enough as a filter, the *:artifactId is probably the fastest though if you are looking for a specific dependency.

如果您对列表也按字母顺序排列（在许多情况下非常方便），然后以下内容也可能有所帮助：

If you are interested in a list of dependencies (not as a tree) also alphabetically ordered (quite handy in many scenarios), then the following may also help:

mvn dependency:list -Dsort=true -DincludeGroupIds=groupId

问题3：当发生依赖性解析时，maven如何使用着色jar中的库？和其他任何一样？

question 3: How does maven use the libraries in shaded jars, when dependency resolution occurs? Same as any other?

通过带阴影的罐子你可能意味着：

By shaded jars you may mean:

• 胖罐，它还将其他罐子带入类路径。在这种情况下，它们被视为一个依赖项，一个单位用于 Maven依赖中介，其内容将成为项目类路径的一部分。一般情况下，您不应该将胖子作为依赖项的一部分，因为您无法控制它带来的打包库。


• 带阴影（重命名）的罐子封装即可。在这种情况下 - 再次 - 就Maven依赖中介而言，没有任何控制权：它是一个单位，一个jar，基于其GAVC（GroupId，ArtifactId，Version，Classifier），使其独一无二。它的内容然后被添加到项目类路径中（根据依赖性范围，但是因为它的包被重命名，你可能有冲突难以处理。再次，你不应该重命名包作为项目依赖项的一部分（但通常你不知道）。

• fat jars, which also bring it other jars into the classpath. In this case, they are seen as one dependency, one unit for Maven Dependency Mediation, its content would then be part of the project classpath. In general, you shouldn't have fat-jars as part of your dependencies since you don't have control over packed libraries it brings in.
• jars with shaded (renamed) packages. In this case - again - there is no control as far as Maven Dependency Mediation is concerned: it's one unit, one jar, based on its GAVC (GroupId, ArtifactId, Version, Classifier) which makes it unique. Its content then it's added to the project classpath (according to the dependency scope, but since its package was renamed, you may have conflicts difficult to handle with. Again, you shouldn't have renamed packages as part of your project dependencies (but often you can't know that).

是否有人拥有他们使用的资源？

Does any one have any resources that they use?

一般来说，你应该很好地理解 Maven如何处理依赖关系并使用它提供的资源（其工具和机制）。以下一些要点：

In general, you should understand well how Maven handles dependencies and use the resources it offers (its tools and mechanisms). Below some important points:

• dependencyManagement 绝对是本主题的入口点：在这里你可以处理Maven Dependency Mediation，影响它对传递依赖的决定封锁，他们的版本，他们的范围。重要的一点是：您添加到 dependencyManagement 的内容不会自动添加为依赖项。 dependencyManagement 只有在项目的某个依赖项（在 pom.xml 文件中声明或通过传递依赖关系）与其中一个条目匹配，否则将被忽略。它是 pom.xml 的重要组成部分，因为它有助于管理依赖关系及其传递图，这就是为什么常常在父poms中使用：你只想处理一个并且在一个集中的方式，你希望在所有Maven项目中使用哪个版本的 log4j ，你在一个公共/共享父pom中声明它及其 dependencyManagement 并确保它将被用作。集中化意味着更好的治理和更好的维护。


• 依赖部分很重要用于声明依赖关系：通常，您应该在此声明直接依赖关系需要。一个很好的重击规则是：在这里声明为编译（默认）范围只是你实际用作 import 语句的范围在你的代码中（但是你经常需要超越它，例如，运行时需要的JDBC驱动程序，而且从未在代码中引用过，它会在 runtime 范围内）。还要记住：声明的顺序很重要：第一个声明的依赖关系会在与传递依赖冲突的情况下获胜，因此通过重新声明依赖关系可以有效地影响依赖关系中介。


• 请勿滥用依赖项中的排除 来处理传递依赖项：使用 dependencyManagement 并订购如果可以的话，依赖项。滥用排除使维护变得更加困难，只有在您真正需要时才使用它。此外，在添加排除项时，请始终添加XML注释，解释原因：您的团队成员或/和您未来的自我将会欣赏。


• 使用依赖项范围若有所思地。使用默认（ compile ）作用域来编译和测试真正需要的内容（例如 loga4j ），使用 test 仅限（且仅限）测试中使用的内容（例如 junit ），请注意提供了目标容器已经提供的范围（例如 servlet-api ），使用运行时范围仅适用于您在运行时所需的内容，但您绝不应该使用它进行编译（例如JDBC驱动程序）。不要使用系统范围，因为它只会带来麻烦（例如它没有与最终工件一起打包）。


• 不要使用版本范围 ，除非由于特定原因并且请注意，默认情况下指定的版本是最低要求， [< version>] 表达式是最强的，但您很少需要它。


• 使用Maven 属性作为版本元素的占位符库的系列，以确保您有一个集中的位置来版本化一组依赖项，这些依赖项都具有相同的版本值。一个典型的例子是用于多个依赖项的 spring.version 或 hibernate.version 属性。同样，集中化意味着更好的治理和维护，这也意味着更少的头痛和更少的地狱。


• 提供时，导入BOM 作为上述要点的替代方案，以便更好地处理依赖家族（例如 jboss ），委托给另一个 pom.xml 归档某些依赖项的管理。


• 不要（ab）使用 SNAPSHOT 依赖（或尽可能少）。如果你真的需要，请确保你永远不会使用 SNAPSHOT 依赖项发布：否则构建再现性将面临很高的危险。


• 进行故障排除时，请始终使用 pom.xml 文件的完整层次结构插件/ maven-help-plugin / effective-pom-mojo.htmlrel =noreferrer> help：effective-pom 在检查时可能非常有用有效 dependencyManagement ，依赖和属性最后的依赖图将被关注。


• 使用其他一些Maven插件来帮助你完成治理。 maven-dependency-plugin 在故障排除过程中非常有用，但 maven-enforcer-plugin 也可以提供帮助。以下是一些值得一提的例子：

• dependencyManagement is definitely the entry point in this topic: here you can deal with Maven Dependency Mediation, influence its decision on transitive dependencies, their versions, their scope. One important point is: what you add to dependencyManagement is not automatically added as a dependency. dependencyManagement is only taken into account once a certain dependency of the project (as declared in the pom.xml file or via transitive dependencies) has a matching with one of its entries, otherwise it would be simply ignored. It's an important part of the pom.xml since it helps on governing dependencies and their transitive graphs and that's why is often used in parent poms: you want to handle only one and in a centralized manner which version of, e.g., log4j you want to use in all of your Maven projects, you declare it in a common/shared parent pom and its dependencyManagement and you make sure it will be used as such. Centralization means better governance and better maintenance.
• dependency section is important for declaring dependencies: normally, you should declare here only the direct dependencies you need. A good rule of thump is: declare here as compile (the default) scope only what you actually use as import statement in your code (but you often need to go beyond that, e.g., JDBC driver required at runtime and never referenced in your code, it would then be in runtime scope though). Also remember: the order of declaration is important: the first declared dependency wins in case of conflict against a transitive dependency, hence by re-declaring esplicitely a dependency you can effectively influence dependency mediation.
• Don't abuse with exclusions in dependencies to handle transitive dependencies: use dependencyManagement and order of dependencies for that, if you can. Abuse of exclusions make maintenance much more difficult, use it only if you really need to. Also, when adding exclusions always add an XML comment explaining why: your team mates or/and your future self will appreciate.
• Use dependencies scope thoughtfully. Use the default (compile) scope for what you really need to for compilation and testing (e.g. loga4j), use test only (and only) for what is used under test (e.g. junit), mind the provided scope for what is already provided by your target container (e.g. servlet-api), use the runtime scope only for what you need at runtime but you should never compile with it (e.g. JDBC drivers). Don't use the system scope since it would only imply troubles (e.g. it is not packaged with your final artifact).
• Don't play with version ranges, unless for specific reasons and be aware that the version specified is a minimum requirements by default, the [<version>] expression is the strongest one, but you would rarely need it.
• use Maven property as placeholder for the version element of families of libraries in order to make sure you have one centralised place for the versioning of a set of dependencies which would all have the same version value. A classic example would be a spring.version or hibernate.version property to use for several dependencies. Again, centralisation means better governance and maintenance, which also means less headache and less hell.
• When provided, import BOM as an alternative to the point above and to better handle families of dependencies (e.g. jboss), delegating to another pom.xml file the management of a certain set of dependencies.
• Don't (ab)use SNAPSHOT dependencies (or as less as possible). If you really need to, make sure you never release using a SNAPSHOT dependency: build reproducibility will be in high danger otherwise.
• When troubleshooting, always check the full hierarchy of your pom.xml file, using help:effective-pom may be really useful while checking for effective dependencyManagement, dependencies and properties as far as the final dependency graph would be concerned.
• Use some other Maven plugins to help you out in the governance. The maven-dependency-plugin is really helpful during troubleshooting, but also the maven-enforcer-plugin comes to help. Here are few examples worth to mention:

以下示例将确保没有人（您，您的团队成员，您自己的未来） ）将能够在 compile 范围内添加一个众所周知的测试库：构建将失败。它确保 junit 永远不会达到PROD（包装在您的 war 中，例如）

The following example will make sure that no one (you, your team mates, your future yourself) will be able to add a well-known test library in compile scope: the build will fail. It makes sure junit will never reach PROD (packaged with your war, e.g.)

<plugin>
    <artifactId>maven-enforcer-plugin</artifactId>
    <version>1.4.1<.version>
    <executions>
        <execution>
            <id>enforce-test-scope</id>
            <phase>validate</phase>
            <goals>
                <goal>enforce</goal>
            </goals>
            <configuration>
                <rules>
                    <bannedDependencies>
                        <excludes>
                            <exclude>junit:junit:*:*:compile</exclude>
                            <exclude>org.mockito:mockito-*:*:*:compile</exclude>
                            <exclude>org.easymock:easymock*:*:*:compile</exclude>
                            <exclude>org.powermock:powermock-*:*:*:compile</exclude>
                            <exclude>org.seleniumhq.selenium:selenium-*:*:*:compile</exclude>
                            <exclude>org.springframework:spring-test:*:*:compile</exclude>
                            <exclude>org.hamcrest:hamcrest-all:*:*:compile</exclude>
                        </excludes>
                        <message>Test dependencies should be in test scope!</message>
                    </bannedDependencies>
                </rules>
                <fail>true</fail>
            </configuration>
        </execution>
    </executions>
</plugin>

看看其他标准规则：在出现错误情况时，许多可能对破坏构建很有用：

Have a look at other standard rules this plugin offers: many could be useful to break the build in case of wrong scenarios:

• 您可以禁止依赖（即使过渡性），在很多情况下非常方便


• 你可以在 SNAPSHOT 的情况下失败，在发布配置文件中很方便，作为示例。

• you can ban a dependency (even transitively), really handy in many cases
• you can fail in case of SNAPSHOT used, handy in a release profile, as an example.

再次，一个常见的父pom 可能包含多个这些机制（ dependencyManagement ，enforcer插件，依赖项系列的属性）并确保certa规则受到尊重。您可能无法涵盖所有​​可能的情况，但它肯定会降低您感知和体验的地狱的程度。

Again, a common parent pom could include more than one of these mechanisms (dependencyManagement, enforcer plugin, properties for dependency families) and make sure certain rules are respected. You may not cover all the possible scenarios, but it would definitely decrease the degree of hell you perceive and experience.

这篇关于用Maven处理依赖地狱的系统方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！