SBT 不必要的重新编译 [英] Unnecessary recompilations by SBT

问题描述

我有一个关于 SBT 看似不必要的重新编译的问题.我有以下场景:我在 docker 容器中运行 SBT，通过将带有我的应用程序源代码的卷附加到容器，并以 sbt 作为入口点启动容器.如果我在该容器内连续运行 SBT，它不会重新编译整个应用程序，这很好.

I have a question about seemingly unnecessary recompilations by SBT. I have the following scenario: I’m running SBT inside a docker container, by attaching the volume with my application source code to the container, and launching the container with sbt as entry point. If I continuously run SBT inside that container, it doesn’t recompile whole app, which is good.

但是，如果我在 OS X 上本地启动 SBT，它会进行完整的重新编译.如果之后我在 docker 中再次启动它，它会再次进行完整的重新编译.这需要很长时间，而且真的很烦人.这种行为的原因是什么?

However, if I start SBT natively on OS X, it does the full recompilation. If after that I start it again inside docker, it again does the full recompilation. It takes very long, and is really annoying. What could be the reason for such behavior?

以下是我在容器中启动 SBT 的方法:

Here's how I launch the SBT in the container:

docker run --name=bla -it --net=host -v /Users/me/.ivy2:/tmp/.ivy2 \
-v /Users/me/.aws/config:/root/.aws/config \
-v /Users/me/.sbt:/root/.sbt \
-v /Users/me/projects/myapp:/src 01ac0b888527 \
/bin/sh -c 'sbt -Dsbt.ivy.home=/tmp/.ivy2 -Divy.home=/tmp/.ivy2 -jvm-debug 5005 -mem 3072'

我的 Java、Scala 和 SBT 版本在主机和容器中是相同的.具体:Scala 2.11.8、Java 1.8.0_77、SBT 0.13.11

My Java, Scala and SBT versions are the same on the host and in the container. Concretely: Scala 2.11.8, Java 1.8.0_77, SBT 0.13.11

推荐答案

好的，经过一天的调试，我找到了解决这个问题的方法.

Okay, after a day of debugging I've found the way around this problem.

SBT 主要基于以下规则使编译的类无效:

SBT invalidates the compiled classes mainly based on the following rules:

• 规范路径
• 最后修改日期

也就是说，路径和修改日期必须完全相同

That is, paths and modification dates have to be exactly same for

• 源代码文件
• 常春藤依赖 jars
• JRE 的罐子

前 2 点很容易实现，因为它只是 docker 卷的映射.关键是映射到与主机上完全相同的路径.例如，如果您像我一样在 OS X 上工作，您的项目源路径可能如下所示:/Users//projects/bla，因此在您的 docker run 命令中，您必须做类似的事情:

First 2 points are quite easy to achieve, because it's just mapping of docker volumes. The crucial thing is to map to exactly same path as on the host machine. For example, if you work on OS X as I do, path your project sources probably looks like this: /Users/<username>/projects/bla, so in your docker run command you have to do something like:

docker run ... -v /Users/<username>/projects/bla:/Users/<username>/projects/bla ...

您不关心源代码和 ivy jar 的时间戳，因为它们将完全相同(它们是相同的文件).

You don't care about timestamps for sources and ivy jars, because they will be exactly same (it's the same files).

您关心时间戳的地方是 JRE 的东西.我用 JRE 构建了 docker 镜像(使用 sbt-docker 插件)，所以我最终读取了本地 JRE 库的修改日期并在镜像中设置了相同的日期:

Where you have to care about timestamps is the JRE stuff. I build the docker image with JRE baked in (using sbt-docker plugin), so I ended up reading modification date of local JRE libs and setting same dates inside the image:

new mutable.Dockerfile {
  ...
  val hostJreTimestamp = new Date(new File(javaHome + "/jre/lib/rt.jar").lastModified()).toString
  val hostJceTimestamp = new Date(new File(javaHome + "/jre/lib/jce.jar").lastModified()).toString
  runRaw(s"""touch -d "$hostJreTimestamp" $javaHome/jre/lib/rt.jar""")
  runRaw(s"""touch -d "$hostJceTimestamp" $javaHome/jre/lib/jce.jar""")
  ...
}

当然，JRE 也应该安装到与主机上完全相同的路径，例如，如果您曾经从 RPM 安装 Java，这可能会出现问题.我最终下载了服务器 JRE(作为 .tar.gz 分发)并手动将其解压缩到正确的路径.

And, of course, JRE should also be installed to exactly same path as on the host, which might be problematic if you used to install Java from RPM, for example. I ended up downloading server JRE (which is distributed as .tar.gz) and extracting it to the right path manually.

所以，长话短说，它最终奏效了.无需重新编译，无需漫长的等待时间.我能够从 2 个主要来源找到相关信息:SBT 源代码，尤其是这个函数:https://github.com/sbt/sbt/blob/0.13/compile/inc/src/main/scala/sbt/inc/IncrementalCommon.scala#L271，并在 build.sbt 中启用 SBT 调试输出:

So, long story short, it worked in the end. No recompilation, no long waiting time. I was able to find the relevant information from 2 main sources: SBT source code, particularly this function: https://github.com/sbt/sbt/blob/0.13/compile/inc/src/main/scala/sbt/inc/IncrementalCommon.scala#L271, and enabling SBT debug output in build.sbt:

logLevel := Level.Debug
incOptions ~= { _.copy(apiDebug = true, relationsDebug = true) }

(准备大量输出)

这篇关于SBT 不必要的重新编译的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！