Docker 中的 .NET 包还原与构建分开缓存 [英] .NET package restore in Docker cached separately from build
问题描述
如何构建 .NET 5/C# 应用程序的 Docker 映像，以便正确缓存还原的 NuGet 包? 正确缓存是指当源(但不是项目文件)是更改后，在 docker build 期间仍会从缓存中获取包含已恢复包的层.
How does one build a Docker image of a .NET 5/C# app so that the restored NuGet packages are cached properly? By proper caching I mean that when sources (but not project files) are changed, the layer containing restored packages is still taken from cache during docker build.
在添加完整源和构建应用程序本身之前执行包还原是 Docker 中的最佳实践，因为它可以单独缓存还原，从而显着加快构建速度.我知道不仅 packages 目录，而且各个项目的 bin 和 obj 目录都必须从 dotnet restore 中保留 到 dotnet publish --no-restore 以便一切正常.我也知道一旦缓存被破坏，所有后续层都会重新构建.
It is a best practice in Docker to perform package restore before adding the full sources and building the app itself as it makes it possible to cache the restore separately, which significantly speeds up the builds. I know that not only the packages directory, but also the bin and obj directories of individual projects have to be preserved from dotnet restore to dotnet publish --no-restore so that everything works together. I also know that once the cache is busted, all following layers are built anew.
我的问题是我无法想出只复制 *.csproj 的方法.如果我复制的不仅仅是 *.csproj，源更改会破坏缓存.我可以将它们复制到 docker build 之外的一个地方，然后简单地将它们复制到构建中，但我希望能够手动甚至在管道之外构建图像，用一个相当简单的命令.(这是一个不合理的要求吗?)
My issue is that I cannot come up with a way to COPY just the *.csproj. If I copy more than just the *.csproj, source changes bust the cache. I could copy them into one place outside the docker build and simply COPY them inside the build, but I want to be able to build the image even outside the pipeline, manually, with a reasonably simple command. (Is it an unreasonable requirement?)
对于在非常标准的文件夹结构中包含多个项目的 Web 应用程序 src/*/*.csproj，我想出了这个尝试来补偿被复制到的太多文件图像(仍然破坏缓存):
For the web app that consists of multiple projects in a pretty standard folder structure src/*/*.csproj, I came up with this attempt that tries to compensate for too many files being copied into the image (which still busts the cache):
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build-env
WORKDIR /src
COPY NuGet.Config NuGet.Config
COPY src/ src/
RUN find . -name NuGet.Config -prune -o ! -type d ! -name *.csproj -exec rm -f '{}' +
&& find -depth -type d -empty -exec rmdir '{}' ;
RUN dotnet restore src/Company.Product.Component.App/Company.Product.Component.App.csproj
COPY src/ src/
RUN dotnet publish src/Company.Product.Component.App/Company.Product.Component.App.csproj -c Release --no-restore -o /out
FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS run-env
WORKDIR /app
COPY --from=build-env /out .
ENTRYPOINT ["dotnet", "Company.Product.Component.App.dll"]
我还尝试在恢复后将 build-env 阶段拆分为两个，将/root/.nuget/packages 和/src 复制到构建阶段，但这也无济于事.
I also tried splitting the build-env stage into two just after the restore, copying the /root/.nuget/packages and /src to the build stage, but that did not help either.
第一个 RUN 行和之前的那个应该被替换为只复制 *.csproj 的东西，但我不知道那是什么.明显费力的解决方案是为每个 *.csproj 设置一个单独的 COPY 行，但这感觉不对，因为项目往往会被添加和删除，因此它使 Dockerfile很难维护.我试过 COPY src/*/*.csproj src/ 然后修复扁平路径，这是我用谷歌搜索的一个技巧，但它对我不起作用，因为我的 Docker 只是在处理通配符文件名并逐字解释目录名，对于不存在的 src/* 目录发出错误.我正在使用 Docker Desktop 3.5.2 (66501)，它使用 BuildKit 后端来构建图像，但如果有帮助，我愿意更改工具.
The first RUN line and the one immediately before should be replaced with something that copies just the *.csproj, but I don't know what that is. The obvious laborious solution is to have a separate COPY line for each *.csproj, but that does not feel right as projects tend to be added and removed so it makes the Dockerfile hard to maintain. I have tried COPY src/*/*.csproj src/ and then fixing the flattened paths, which is a trick that I googled, but it did not work for me as my Docker processes the wildcards just in file names and interprets directory names literally, emitting an error for nonexistent src/* directory. I am using Docker Desktop 3.5.2 (66501), which uses the BuildKit backend to build the images, but I am open to changing the tooling if it helps.
这让我对如何满足我的一组相对简单的要求一无所知.我的选择似乎用尽了.我错过了什么吗?我是否必须接受权衡并放弃一些要求?
This leaves me clueless about how to satisfy the relatively simple set of my requirements. My options seem exhausted. Have I missed something? Do I have to accept a tradeoff and drop some of my requirements?
推荐答案
缺少对目录名称中通配符的支持可能是 BuildKit 中缺少的功能.该问题已在 moby/buildkit GitHub 上报告为 #1900.
The lack of support for wildcards in directory names is likely a missing feature in BuildKit. The issue has already been reported at moby/buildkit GitHub as #1900.
直到问题得到解决，禁用 BuildKit 如果您不需要任何它的特点.要么
Till the issue is fixed, disable BuildKit if you don't need any of its features. Either
- 将环境变量 DOCKER_BUILDKIT 设置为零 (
0)，或者 - 编辑 Docker 守护进程配置，以便buildkit"功能设置为 false 并重新启动守护程序.
在 Docker Desktop 中，可以在设置"中轻松访问配置 >Docker 引擎.推荐使用这种关闭功能的方法Docker Desktop 3.2.0 发行说明，其中 BuildKit 在默认情况下首次启用.
In Docker Desktop, the config is easily accessible in Settings > Docker Engine. This method of turning off the feature is recommended by the Docker Desktop 3.2.0 release notes where BuildKit was first enabled by default.
一旦 BuildKit 被禁用，替换
Once BuildKit is disabled, replace
COPY src/ src/
RUN find . -name NuGet.Config -prune -o ! -type d ! -name *.csproj -exec rm -f '{}' +
&& find -depth -type d -empty -exec rmdir '{}' ;
与
COPY src/*/*.csproj src/
RUN for from in src/*.csproj; do to=$(echo "$from" | sed 's//([^/]*).csproj$//1&/')
&& mkdir -p "$(dirname "$to")" && mv "$from" "$to"; done
COPY 将成功而不会破坏缓存，而 RUN 将修复路径.它依赖于项目位于src"中的事实.目录，每个目录都在与项目文件同名的单独目录中.
The COPY will succeed without busting the cache and the RUN will fix the paths. It relies on the fact the projects are in the "src" directory, each in a separate directory of the same name as the project file.
这基本上是 VonC 对相关问题的回答底部的解决方案.答案还提到了 Moby 问题 #15858，其中对该主题进行了有趣的讨论.
This is basically the solution at the bottom of VonC's answer to a related question. The answer also mentions Moby issue #15858, which has interesting discussion on the topic.
有一个 dotnet 工具用于路径修复也是如此，但我还没有测试过.
There is a dotnet tool for the paths fixup, too, but I have not tested it.
不需要禁用 BuildKit 的替代解决方案是在清理复制的文件后立即将原始阶段一分为二，即在恢复之前(而不是之后！).
An alternate solution that does not require disabling BuildKit is to split the original stage in two right after cleaning up the copied files, i.e. just before before restore (not after!).
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS projects-env
WORKDIR /src
COPY NuGet.Config NuGet.Config
COPY src/ src/
RUN find . -name NuGet.Config -prune -o ! -type d ! -name *.csproj -exec rm -f '{}' +
&& find . -depth -type d -empty -exec rmdir '{}' ;
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build-env
WORKDIR /src
COPY --from=projects-env /src /src
RUN dotnet restore src/Company.Product.Component.App/Company.Product.Component.App.csproj
COPY src/ src/
RUN dotnet publish src/Company.Product.Component.App/Company.Product.Component.App.csproj -c Release --no-restore -o /out
FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS run-env
WORKDIR /app
COPY --from=build-env /out .
ENTRYPOINT ["dotnet", "Company.Product.Component.App.dll"]
sources-env 中的 COPY src/src/ 层因源更改而失效，但缓存失效对每个阶段单独工作.由于复制到 build-env 的文件在构建中是相同的，COPY --from=projects-env 缓存不会失效，因此 RUN dotnet restore 层也从缓存中取出.
The COPY src/ src/ layer in the sources-env is invalidated by source changes, but cache invalidation works separately for each stage. As the files copied over to the build-env are the same across builds, the COPY --from=projects-env cache is not invalidated, so the RUN dotnet restore layer is taken from the cache, too.
我怀疑还有其他使用 BuildKit 的解决方案挂载(RUN --mount=...)，但我还没有测试过.
I suspect there are other solutions using the BuildKit mounts (RUN --mount=...), but I haven't tested any.
这篇关于Docker 中的 .NET 包还原与构建分开缓存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
