是否有一种方法可以将更改的文件添加到码头图像作为新图层 - 而不使用docker提交？ [英] Is there a way to add only changed files to a docker image as a new layer - without resorting to docker commit?
问题描述
TL; DR
运行 COPY。 / app 在图像之上，但稍微过时的源代码创建了一个与整个源代码一样大的新图层，即使只有几个字节的值更改。
有没有办法只将更改的文件添加到这个docker图像中作为一个新的层 - 而不使用docker commit？
Running COPY . /app on top of an image with but slightly outdated source code creates a new layer as large as the whole source code, even when there is only a few bytes worth of changes.
Is there a way to add only changed files to this docker image as a new layer - without resorting to docker commit?
长版本： / strong>
Long version:
当我们将应用程序部署到生产中时，我们需要将源代码添加到图像中。一个非常简单的Dockerfile用于此：
When deploying our application to production, we need to add the source code to the image. A very simple Dockerfile is used for this:
FROM neam/dna-project-base-debian-php:0.6.0
COPY . /app
由于源代码庞大（1.2 GB），这使得相当大的推动力每个部署：
Since the source code is huge (1.2 GB), this makes for quite a hefty push upon each deploy:
$ docker build -f .stack.php.Dockerfile -t project/project-web-src-php:git-commit-17c279b .
Sending build context to Docker daemon 1.254 GB
Step 0 : FROM neam/dna-project-base-debian-php:0.6.0
---> 299c10c416fc
Step 1 : COPY . /app
---> 78a30802804a
Removing intermediate container 13b49c323bb6
Successfully built 78a30802804a
$ docker tag -f project/project-web-src-php:git-commit-17c279b tutum.co/project/project-web-src-php:git-commit-17c279b
$ docker login --email=tutum-project@project.com --username=project --password=******** https://tutum.co/v1
WARNING: login credentials saved in /home/dokku/.docker/config.json
Login Succeeded
$ docker push tutum.co/project/project-web-src-php:git-commit-17c279b
The push refers to a repository [tutum.co/project/project-web-src-php] (len: 1)
Sending image list
Pushing repository tutum.co/project/project-web-src-php (1 tags)
Image a604b236bcde already pushed, skipping
Image 1565e86129b8 already pushed, skipping
...
Image 71156b357f2f already pushed, skipping
Image 299c10c416fc already pushed, skipping
78a30802804a: Pushing [=========> ] 234.2 MB/1.254 GB
下次部署时，我们只想将更改的文件添加到图像，但在运行 COPY时观看和观看。 / app 在先前添加的图像之上实际上需要我们推送1.2 GB的源代码AGAIN，即使我们只更改了几个字节的源代码：
Upon the next deploy, we only want to add the changed files to the image, but watch and behold when running COPY . /app on top of the previously added image actually requires us to push 1.2 GB worth of source code AGAIN, even when we only change a few bytes worth of source code:
新的Docker文件（ .stack.php.git-commit-17c279b.Dockerfile ）：
New Dockerfile (.stack.php.git-commit-17c279b.Dockerfile):
FROM project/project-web-src-php:git-commit-17c279b
COPY . /app
更改几个文件后，添加一些文本和代码，然后构建并推送： p>
After change a few files, adding some text and code, then building and pushing:
$ docker build -f .stack.php.git-commit-17c279b.Dockerfile -t project/project-web-src-php:git-commit-17c279b-with-a-few-changes .
Sending build context to Docker daemon 1.225 GB
Step 0 : FROM project/project-web-src-php:git-commit-17c279b
---> 4dc643a45de3
Step 1 : COPY . /app
---> ecc7adc194c4
Removing intermediate container cb3e87c6cb7a
Successfully built ecc7adc194c4
$ docker tag -f project/project-web-src-php:git-commit-17c279b-with-a-few-changes tutum.co/project/project-web-src-php:git-commit-17c279b-with-a-few-changes
$ docker push tutum.co/project/project-web-src-php:git-commit-17c279b-with-a-few-changes
The push refers to a repository [tutum.co/project/project-web-src-php] (len: 1)
Sending image list
Pushing repository tutum.co/project/project-web-src-php (1 tags)
Image 1565e86129b8 already pushed, skipping
Image a604b236bcde already pushed, skipping
...
Image fe64bff23cf8 already pushed, skipping
Image 71156b357f2f already pushed, skipping
ecc7adc194c4: Pushing [==> ] 68.21 MB/1.225 GB
有一个解决方法来实现小层，如使用提交更新使用小更改的泊坞站图像，其中包括在内部启动rsync进程图像，然后使用docker提交将新内容保存为新层，但是（如该线程中所述），这是非正统的，因为图像不是由Dockerfile构建的，我们更喜欢不依赖docker提交的正统解决方案。
There is a workaround to achieve small layers as described on Updating docker images with small changes using commits which includes launching a rsync process within the image and then using docker commit to save the new contents as a new layer, however (as mentioned in that thread) this is unorthodox since the image is not built from a Dockerfile, and we prefer an orthodox solution that does not rely on docker commit.
是否有一种方法可以将更改的文件添加到此docker图像中作为新图层 - 而不使用docker提交？
Is there a way to add only changed files to this docker image as a new layer - without resorting to docker commit?
Docker版本1.8.3
Docker version 1.8.3
推荐答案
其实，解决方案使用 COPY。 / app ，因为OP正在做，但是一个开放的错误，导致在大多数系统上不能按预期方式工作
Actually, the solution IS to use COPY . /app as the OP is doing, there is however an open bug causing this not to work as expected on most systems
此问题的唯一可行的解决方法似乎是使用rsync分析新旧图像之前的差异，然后使用changelog输出生成一个包含相关更改的tar文件，随后COPY：编辑到新的图像层。
The only currently feasible workaround to this issue seems to be to use rsync to analyze the differences between the old and new images prior to pushing the new one, then use the changelog output to generate a tar-file containing the relevant changes which is subsequently COPY:ed to a new image layer.
这样一来，层次大小就变成了几个字节，也就是千字节，而不是每次1.2 GB。
This way, the layer sizes becomes a few bytes or kilobytes for smaller changes instead of 1.2 GB every time.
我将文档和脚本放在一起，以便在 https://github.com/neam/docker-diff-based-layers 。
I put together documentation and scripts to help out with this over at https://github.com/neam/docker-diff-based-layers.
最终结果如下所示：
验证后续的 COPY。 / app 命令重新添加每个图层中的所有文件，而不是仅更改已更改的文件：
Verify that subsequent COPY . /app commands re-adds all files in every layer instead of only the files that have changed:
docker history sample-project:revision-2
输出：
IMAGE CREATED CREATED BY SIZE COMMENT
4a3115eaf267 3 seconds ago /bin/sh -c #(nop) COPY dir:61d102421e6692b677 16.78 MB
d4b30af167f4 25 seconds ago /bin/sh -c #(nop) COPY dir:68b8f374d8731b8ad8 16.78 MB
c898fe1daa44 2 minutes ago /bin/sh -c apt-get update && apt-get install 10.77 MB
39a8a358844a 4 months ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0 B
b1dacad9c5c9 4 months ago /bin/sh -c #(nop) ADD file:5afd8eec1dc1e7666d 125.1 MB
即使我们只添加了几个字节，文件被重新添加，16.78 MB被添加到总图像大小。
Even though we added/changed only a few bytes, all files are re-added and 16.78 MB is added to the total image size.
此外，我们删除的文件没有被删除。
Also, the file(s) that we removed did not get removed.
export RESTRICT_DIFF_TO_PATH=/app
export OLD_IMAGE=sample-project:revision-1
export NEW_IMAGE=sample-project:revision-2
docker-compose -f rsync-image-diff.docker-compose.yml up
docker-compose -f shell.docker-compose.yml -f process-image-diff.docker-compose.yml up
cd output; docker build -t sample-project:revision-2-processed .; cd ..
验证处理后的新图像是否具有更小的图层，并具有更改：
Verify that the processed new image has smaller sized layers with the changes:
docker history sample-project:revision-2-processed
输出：
IMAGE CREATED CREATED BY SIZE COMMENT
1920e750d362 24 seconds ago /bin/sh -c if [ -s /.files-to-remove.list ]; 0 B
1267bf926729 2 minutes ago /bin/sh -c #(nop) ADD file:5021c627243e841a45 19 B
d04a2181b62a 2 minutes ago /bin/sh -c #(nop) ADD file:14780990c926e673f2 264 B
d4b30af167f4 7 minutes ago /bin/sh -c #(nop) COPY dir:68b8f374d8731b8ad8 16.78 MB
c898fe1daa44 9 minutes ago /bin/sh -c apt-get update && apt-get install 10.77 MB
39a8a358844a 4 months ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0 B
b1dacad9c5c9 4 months ago /bin/sh -c #(nop) ADD file:5afd8eec1dc1e7666d 125.1 MB
验证处理后的新图像是否包含与原始：
Verify that the processed new image contains the same contents as the original:
export RESTRICT_DIFF_TO_PATH=/app
export OLD_IMAGE=sample-project:revision-2
export NEW_IMAGE=sample-project:revision-2-processed
docker-compose -f rsync-image-diff.docker-compose.yml up
输出应该表示图像/标签之间没有差异。因此，现在可以推送和部署sample-project：revision-2-processed标签，导致相同的最终结果，但无需通过电线推送不必要的16.78M，从而实现更快的部署周期。
The output should indicate that there are no differences between the images/tags. Thus, the sample-project:revision-2-processed tag can now be pushed and deployed, leading to the same end result but without having to push an unnecessary 16.78M over the wire, leading to faster deploy cycles.
这篇关于是否有一种方法可以将更改的文件添加到码头图像作为新图层 - 而不使用docker提交？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
