Docker 增量构建不重用缓存 [英] Docker incremental build does not reuse cache
本文介绍了Docker 增量构建不重用缓存的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我正在尝试通过应用此处解释的增量构建原则以优化的方式构建一个巨大的 docker 镜像 https://www.docker.com/blog/intro-guide-to-dockerfile-best-practices/ .
I am trying to build a huge docker image in an optimized way by applying the principles of incremental building explained here https://www.docker.com/blog/intro-guide-to-dockerfile-best-practices/ .
不幸的是，每次运行构建命令时，docker 都会从头开始构建映像，因此我必须再次下载所有 maven 依赖项.
Unfortunately each time I run the build command docker restarts building the image from scratch, and so I have to download again all the maven dependencies.
这是构建命令:
docker build \
--build-arg MAVEN_SETTINGS_FILE="${HOME}/.m2/settings.xml" \
--build-arg PROJECT_PATH="." \
--file "${DOCKER_FILE}" \
--tag "${IMAGE_TAG}" \
.
这里是 Dockerfile:
# Global vars (for passing between stages)
ARG MAVEN_SETTINGS_FILE
ARG PROJECT_PATH
ARG APP_FOLDER=/app
# 1st stage
FROM maven:3.6-jdk-11 as build
# Build artifact
ARG APP_FOLDER
ENV APP_FOLDER ${APP_FOLDER}
ARG MAVEN_SETTINGS_FILE
ENV MAVEN_SETTINGS_FILE ${MAVEN_SETTINGS_FILE}
ARG PROJECT_PATH
ENV PROJECT_PATH ${PROJECT_PATH}
WORKDIR ${APP_FOLDER}
ADD ${MAVEN_SETTINGS_FILE} .
ADD ${PROJECT_PATH}/pom.xml .
RUN mvn package -s ${MAVEN_SETTINGS_FILE} -DskipTests
ADD ${PROJECT_PATH}/src src
RUN mvn package -s ${MAVEN_SETTINGS_FILE} -DskipTests
# 2nd stage: build image
FROM openjdk:11-jre-slim
ARG APP_FOLDER
ENV APP_FOLDER ${APP_FOLDER}
ENV TARGET_FOLDER ${APP_FOLDER}/target
WORKDIR ${APP_FOLDER}
# Copy the binary built in the 1st stage
COPY --from=build ${TARGET_FOLDER}/myapp-1.0.0-SNAPSHOT.jar .
EXPOSE 8080
CMD ["java", "-jar", "myapp-1.0.0-SNAPSHOT.jar"]
非常感谢您的任何建议！
Thank you so much for any suggestion!
嗨@sai，我通过docker history 命令检查了缓存层，发现缺少MVN 步骤！
Hi @sai, I checked the cached layers by docker history command and I observed that the MVN steps are missing!
IMAGE CREATED CREATED BY SIZE COMMENT
b78b3d09d314 16 minutes ago /bin/sh -c #(nop) CMD ["java" "-Doracle.jdb… 0B
a2db7da187f0 16 minutes ago /bin/sh -c #(nop) EXPOSE 8080 0B
ed8f3dc45017 16 minutes ago /bin/sh -c #(nop) EXPOSE 8081 0B
715f12eba281 16 minutes ago /bin/sh -c #(nop) COPY dir:33e2303bf32b392fc… 58.3MB
4edb35a1b6f6 16 minutes ago /bin/sh -c #(nop) COPY file:0b42aca8bb0ad316… 234kB
56ecbfb34e74 16 minutes ago /bin/sh -c #(nop) COPY dir:8497884af419f408f… 4.59kB
255913b0fc25 16 minutes ago /bin/sh -c #(nop) WORKDIR /app 0B
87c5b4ca34df 16 minutes ago /bin/sh -c #(nop) ENV TARGET_FOLDER=/app/ta… 0B
5f95ab5b8d19 16 minutes ago /bin/sh -c #(nop) ENV APP_FOLDER=/app 0B
da6d51b1d3b8 16 minutes ago /bin/sh -c #(nop) ARG APP_FOLDER 0B
940f48bb6c92 3 days ago /bin/sh -c set -eux; arch="$(dpkg --print-… 142MB
<missing> 3 days ago /bin/sh -c #(nop) ENV JAVA_VERSION=11.0.11+9 0B
<missing> 3 days ago /bin/sh -c #(nop) ENV LANG=C.UTF-8 0B
<missing> 3 days ago /bin/sh -c #(nop) ENV PATH=/usr/local/openj… 0B
<missing> 3 days ago /bin/sh -c { echo '#/bin/sh'; echo 'echo "$J… 27B
<missing> 3 days ago /bin/sh -c #(nop) ENV JAVA_HOME=/usr/local/… 0B
<missing> 3 days ago /bin/sh -c set -eux; apt-get update; apt-g… 8.82MB
<missing> 3 days ago /bin/sh -c #(nop) CMD ["bash"] 0B
<missing> 3 days ago /bin/sh -c #(nop) ADD file:7362e0e50f30ff454… 69.3MB
我的观点是，所有使用要扩展的变量的指令都不会生成缓存层.你也适合吗?
My opinion is that all the instructions which use a variable to be expanded do not generate a cached layer. Is it right for you too?
我尝试使用 Helidon 演示进行挖掘，我观察到它没有在历史记录中显示 MVN 层.相反，它使用缓存层.
I tryed to go digger with Helidon demo, and I observed that it does not show in the history the MVN layer. Conversely it uses cached layers.
查看docker文件和第二个构建执行日志:
Look at the docker file and second build execution log:
# 1st stage, build the app
FROM maven:3.6-jdk-11 as build
WORKDIR /helidon
# Create a first layer to cache the "Maven World" in the local repository.
# Incremental docker builds will always resume after that, unless you update
# the pom
ADD pom.xml .
RUN mvn package -Dmaven.test.skip -Declipselink.weave.skip
# Do the Maven build!
# Incremental docker builds will resume here when you change sources
ADD src src
RUN mvn package -DskipTests
RUN echo "done!"
# 2nd stage, build the runtime image
FROM openjdk:11-jre-slim
WORKDIR /helidon
# Copy the binary built in the 1st stage
COPY --from=build /helidon/target/helidon-quickstart-mp.jar ./
COPY --from=build /helidon/target/libs ./libs
CMD ["java", "-jar", "helidon-quickstart-mp.jar"]
EXPOSE 8080
Sending build context to Docker daemon 43.01kB
Step 1/13 : FROM maven:3.6-jdk-11 as build
---> e23b595c92ad
Step 2/13 : WORKDIR /helidon
---> Using cache
---> 25e45ff1f01d
Step 3/13 : ADD pom.xml .
---> Using cache
---> ec5c0a3ecd2c
Step 4/13 : RUN mvn package -Dmaven.test.skip -Declipselink.weave.skip
---> Using cache
---> a21083c406a0
Step 5/13 : ADD src src
---> Using cache
---> d718f90a4c6d
Step 6/13 : RUN mvn package -DskipTests
---> Using cache
---> ebdb2ff847fd
Step 7/13 : RUN echo "done!"
---> Using cache
---> d9c9f46d0af2
Step 8/13 : FROM openjdk:11-jre-slim
---> 940f48bb6c92
Step 9/13 : WORKDIR /helidon
---> Using cache
---> e0aa150de7c3
Step 10/13 : COPY --from=build /helidon/target/helidon-quickstart-mp.jar ./
---> Using cache
---> 0f64c021b20f
Step 11/13 : COPY --from=build /helidon/target/libs ./libs
---> Using cache
---> d2c6fad54ac1
Step 12/13 : CMD ["java", "-jar", "helidon-quickstart-mp.jar"]
---> Using cache
---> 7de63ada236c
Step 13/13 : EXPOSE 8080
---> Using cache
---> d6a11cd62373
Successfully built d6a11cd62373
Successfully tagged helidon-quickstart-mp:latest
**编辑 3(找到问题原因)
我看了我的第二个构建执行日志，我认为，ENV 变量分配破坏了缓存:
**Edit 3 (issue reason found)
I looked better at my second build execution log and I supposed, the ENV variabiles assignments breaking the caching:
Sending build context to Docker daemon 2.294MB
Step 1/36 : ARG CI_CONTAINER_BUILD_FULL_VERSION
Step 2/36 : ARG CI_CONTAINER_BUILD_PROFILE
Step 3/36 : ARG CI_CONTAINER_BUILD_VERSION
Step 4/36 : ARG CI_CONTAINER_MAVEN_BUILD_OPTIONS
Step 5/36 : ARG CI_CONTAINER_MAVEN_POM_ARTIFACT_ID
Step 6/36 : ARG CI_CONTAINER_MAVEN_POM_GROUP_ID
Step 7/36 : ARG CI_CONTAINER_MAVEN_POM_VERSION
Step 8/36 : ARG CI_CONTAINER_MAVEN_SETTINGS_FILE
Step 9/36 : ARG CI_CONTAINER_PROJECT_PATH
Step 10/36 : ARG APP_FOLDER=/app
Step 11/36 : FROM maven:3.6-jdk-11 as build
---> e23b595c92ad
Step 12/36 : ARG APP_FOLDER
---> Using cache
---> 7418c1e78088
Step 13/36 : ENV APP_FOLDER ${APP_FOLDER}
---> Using cache
---> 9fb55b74b57b
Step 14/36 : ARG CI_CONTAINER_MAVEN_SETTINGS_FILE
---> Using cache
---> 9de7642ea7ae
Step 15/36 : ENV CI_CONTAINER_MAVEN_SETTINGS_FILE ${CI_CONTAINER_MAVEN_SETTINGS_FILE}
---> Running in 0d5655b4dd2b
Removing intermediate container 0d5655b4dd2b
---> d590a3bc4167
Step 16/36 : ARG CI_CONTAINER_MAVEN_BUILD_OPTIONS
---> Running in 528c5d6c82f2
Removing intermediate container 528c5d6c82f2
---> 5ec69a0c4629
Step 17/36 : ENV CI_CONTAINER_MAVEN_BUILD_OPTIONS ${CI_CONTAINER_MAVEN_BUILD_OPTIONS}
---> Running in eefe59ddc94b
Removing intermediate container eefe59ddc94b
---> beb77c7c67e5
Step 18/36 : ARG CI_CONTAINER_PROJECT_PATH
---> Running in da389f2e0824
Removing intermediate container da389f2e0824
---> 6835d33be70e
Step 19/36 : ENV CI_CONTAINER_PROJECT_PATH ${CI_CONTAINER_PROJECT_PATH}
---> Running in 89420b67a110
Removing intermediate container 89420b67a110
---> fd1b05ed1dfd
Step 20/36 : WORKDIR ${APP_FOLDER}
---> Running in 39135509f1d7
Removing intermediate container 39135509f1d7
---> 833258753a45
Step 21/36 : COPY ${CI_CONTAINER_MAVEN_SETTINGS_FILE} .
---> 6f1b6a7176de
Step 22/36 : COPY ${CI_CONTAINER_PROJECT_PATH}/pom.xml .
---> 92f34751c641
Step 23/36 : RUN mvn -e -B -s ${CI_CONTAINER_MAVEN_SETTINGS_FILE} -DskipTests -Dmaven.openapi-generator.skip ${CI_CONTAINER_MAVEN_BUILD_OPTIONS} package
---> Running in 38fcf8af4523
[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO] Downloading from public: http://ci.betting.sisal.it/nexus/repository/maven-public/it/sisal/betting/root-pom/2.0.2/root-pom-2.0.2.pom
[INFO] Downloaded from public: http://ci.betting.sisal.it/nexus/repository/maven-public/it/sisal/betting/root-pom/2.0.2/root-pom-2.0.2.pom (12 kB at 41 kB/s)
[INFO] Downloading from public: http://ci.betting.sisal.it/nexus/repository/maven-public/io/helidon/helidon-dependencies/2.3.0/helidon-dependencies-2.3.0.pom
... and so on...
现在我应该知道让变量赋值可缓存！！！
推荐答案
我找到了解决方案！
第一个问题与我在原始帖子中没有显示的我的构建命令的细节有关:MAVEN_SETTINGS_FILE 是由 mktemp 生成的，所以它曾经在任何执行时更改，从而使以下层无效.
The first problem was related to a detail of my build command that I did not show in the original post: the MAVEN_SETTINGS_FILE was generated by the mktemp so it used to change at any execution, consequently invalidating the following layers.
第二个问题是我并不真正需要 ENV 变量，但 ARG 就足够了.
The second issue was that I did not really need the ENV variables, but the ARGs were enough.
因此，我将它们重写如下，并且所有层都按预期从缓存中重用.
So, I rewritten them as follow, and all the layers are reused from cache as expected.
M2_SETTINGS_FILE=.settings.xml
cp "${HOME}/.m2/settings.xml" "${M2_SETTINGS_FILE}"
docker build \
--build-arg MAVEN_SETTINGS_FILE="${M2_SETTINGS_FILE}" \
--build-arg PROJECT_PATH="." \
--file "${DOCKER_FILE}" \
--tag "${IMAGE_TAG}" \
.
# Global vars (for passing between stages)
ARG MAVEN_SETTINGS_FILE
ARG PROJECT_PATH
ARG APP_FOLDER=/app
# 1st stage
FROM maven:3.6-jdk-11 as build
# Build artifact
ARG APP_FOLDER
ARG MAVEN_SETTINGS_FILE
ARG PROJECT_PATH
WORKDIR ${APP_FOLDER}
ADD ${MAVEN_SETTINGS_FILE} .
ADD ${PROJECT_PATH}/pom.xml .
RUN mvn package -s ${MAVEN_SETTINGS_FILE} -DskipTests
ADD ${PROJECT_PATH}/src src
RUN mvn package -s ${MAVEN_SETTINGS_FILE} -DskipTests
# 2nd stage: build image
FROM openjdk:11-jre-slim
ARG APP_FOLDER
ARG TARGET_FOLDER ${APP_FOLDER}/target
WORKDIR ${APP_FOLDER}
# Copy the binary built in the 1st stage
COPY --from=build ${TARGET_FOLDER}/myapp-1.0.0-SNAPSHOT.jar .
EXPOSE 8080
CMD ["java", "-jar", "myapp-1.0.0-SNAPSHOT.jar"]
这篇关于Docker 增量构建不重用缓存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
