使用--cache-from复制Gemfile时，Docker构建未使用缓存 [英] Docker build not using cache when copying Gemfile while using --cache-from

问题描述

在我的本地计算机上，我已经构建了最新的映像，并且运行另一个 docker build 时，它在各处都使用了缓存。

然后我将图像作为 latest 上载到注册表，然后在我的CI服务器上， 'm拉出我应用程序的最新图片，以便将其用作构建新版本的构建缓存：

  docker pull $ CONTAINER_IMAGE：最新
 
 docker build --cache-from $ CONTAINER_IMAGE：latest \ 
 --tag $ CONTAINER_IMAGE：$ CI_COMMIT_SHORT_SHA b 
。 
  

从构建输出中，我们可以看到 COPY Gemfile 的>没有使用 latest 图像中的大小写，但我没有更新该文件：

 步骤15/22：运行gem install bundler -v 1.17.3&& ln -s /usr/local/lib/ruby/gems/2.2.0/gems/bundler-1.16.0 /usr/local/lib/ruby/gems/2.2.0/gems/bundler-1.16.1 
 --->使用缓存
 ---> 47a9ad7747c6 
步骤16/22：ENV BUNDLE_GEMFILE = $ APP_HOME / Gemfile BUNDLE_JOBS = 8 
 --->使用缓存
 ---> 1124ad337b98 
步骤17/22：WORKDIR $ APP_HOME 
 --->使用缓存
 ---> 9cd742111641 
步骤18/22：复制Gemfile $ APP_HOME / 
 ---> f7ff0ee82ba2 
步骤19/22：复制Gemfile.lock $ APP_HOME / 
 ---> c963b4c4617f 
步骤20/22：RUN bundle install 
 --->在3d2cdf999972中运行
  

旁节点：它在我的本地计算机上运行良好

查看Docker文档利用构建缓存似乎在这里没有解释行为，因为Dockerfile或Gemfile均未更改，因此应使用缓存。

是什么让Docker不使用Gemfile缓存？

更新

我尝试使用 COPY --chown = user：group源dest 复制设置正确权限的文件，但是它仍然不使用缓存。

打开Docker论坛主题： https://forums.docker.com/t/docker-build-not-using-cache-when-copying-gemfile-while-using-cache-from / 69186

解决方案

我一直在努力解决Docker build和的问题--cache-from 的最后几天，对于缺少-cache-from 的正确行为的文档资料的使用，这有点令人沮丧。

我想，在我得到一些见解之后，我终于设法解决了我身边的问题

当提供多个-cache-from ，顺序很重要！

顺序非常重要，因为在第一场比赛中， Docker将停止寻找其他比赛，并将其用于其他比赛所有其余命令。

这是由在Github PR中实现该功能的人解释的：

使用多个--cache-from时，将按照用户指定的顺序检查它们的缓存命中。如果其中一幅图像产生命令命中缓存命中，则只有该图像用于其余的构建。

初始票务提案中的详细解释：

指定多个--cache-from图像有点麻烦。如果两个图像都匹配，则无法（不进行多次扫描）找出要使用的图像。因此，我们选择第一个（让用户控制优先级），但这可能不是最后我们可以匹配的最长链。如果我们允许某些命令与一个映像进行匹配，然后又切换到链较长的另一个映像，则由于我们仅验证历史记录和缓存层，因此有可能泄漏映像之间的某些信息。目前，我已将其保留下来，以便如果我们找到匹配项，则仅将该目标图像用于其余命令。

使用-cache-from 是独占的：不会使用本地Docker缓存

这意味着它不会添加新的缓存源，您提供的图像标签将是Docker构建的唯一缓存源。

即使您只是在本地构建相同的映像，下次运行时docker为此构建，为了从缓存中受益，您需要：




1. 使用<$提供正确的标签c $ c>-cache-from （并具有正确的优先级）；或




2. 根本不使用-cache-from （这样它将使用本地版本缓存）

如果父图像发生更改，则缓存将无效

例如，如果您有一个基于 docker：stable 的映像，而 docker：stable 得到

这就是为什么如果要配置CI构建的原因，那么更新后的图像的缓存构建将不再有效。

，对 docker pull 也将基本图片包含在-cache-from 中，如本注释在Github的另一篇讨论中所述。

On my local machine, I have built the latest image, and running another docker build uses cache everywhere it should.

Then I upload the image to the registry as the latest, and then on my CI server, I'm pulling the latest image of my app in order to use it as the build cache to build the new version :

docker pull $CONTAINER_IMAGE:latest

docker build --cache-from $CONTAINER_IMAGE:latest \
             --tag $CONTAINER_IMAGE:$CI_COMMIT_SHORT_SHA \
             .

From the build output we can see the COPY of the Gemfile is not using the case from the latest image, while I haven't updated that file :

Step 15/22 : RUN gem install bundler -v 1.17.3 &&     ln -s /usr/local/lib/ruby/gems/2.2.0/gems/bundler-1.16.0 /usr/local/lib/ruby/gems/2.2.0/gems/bundler-1.16.1
 ---> Using cache
 ---> 47a9ad7747c6
Step 16/22 : ENV BUNDLE_GEMFILE=$APP_HOME/Gemfile     BUNDLE_JOBS=8
 ---> Using cache
 ---> 1124ad337b98
Step 17/22 : WORKDIR $APP_HOME
 ---> Using cache
 ---> 9cd742111641
Step 18/22 : COPY Gemfile $APP_HOME/
 ---> f7ff0ee82ba2
Step 19/22 : COPY Gemfile.lock $APP_HOME/
 ---> c963b4c4617f
Step 20/22 : RUN bundle install
 ---> Running in 3d2cdf999972

Aside node : It is working perfectly on my local machine.

Looking at the Docker documentation Leverage build cache doesn't seem to explain the behaviour here as nor the Dockerfile, nor the Gemfile have changed, so the cache should be used.

What could make Docker not using the cache for the Gemfile?

Update

I tried to copy the files setting the right permissions using COPY --chown=user:group source dest but it still doesn't use the cache.

Opened Docker forum topic: https://forums.docker.com/t/docker-build-not-using-cache-when-copying-gemfile-while-using-cache-from/69186

解决方案

I've been scratching my head over issues with Docker build and --cache-from for the last few days, and it's a bit frustrating the lack of documentation for the proper behavior of --cache-from, while there is some misinformation in the wild.

I think I've finally managed to fix the issues I had on my side, after a few insights which I'm going to share here in the hopes it will be useful to someone else.

When providing multiple --cache-from, the order matters!

The order is very important, because at the first match, Docker will stop looking for other matches and it will use that one for all the rest of the commands.

This is explained by the fellow who implemented the feature in the Github PR:

When using multiple --cache-from they are checked for a cache hit in the order that user specified. If one of the images produces a cache hit for a command only that image is used for the rest of the build.

There is also a lenghtier explanation in the initial ticket proposal:

Specifying multiple --cache-from images is bit problematic. If both images match there is no way(without doing multiple passes) to figure out what image to use. So we pick the first one(let user control the priority) but that may not be the longest chain we could have matched in the end. If we allow matching against one image for some commands and later switch to a different image that had a longer chain we risk in leaking some information between images as we only validate history and layers for cache. Currently I left it so that if we get a match we only use this target image for rest of the commands.

Using --cache-from is exclusive: the local Docker cache won't be used

This means that it doesn't add new caching sources, the image tags you provide will be the only caching sources for the Docker build.

Even if you just built the same image locally, the next time you run docker build for it, in order to benefit from the cache, you need to either:

1. provide the correct tag with --cache-from (and with the correct precedence); or

2. not use --cache-from at all (so that it will use the local build cache)

If the parent image changes, the cache will be invalidated

For example, if you have an image based on docker:stable, and docker:stable gets updated, the cached builds of your image will not be valid anymore as the layers of the base image were changed.

This is why, if you're configuring a CI build, it can be useful to docker pull the base image as well and include it in the --cache-from, as mentioned in this comment in yet another Github discussion.

这篇关于使用--cache-from复制Gemfile时，Docker构建未使用缓存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！