在同一Docker Hub存储库中进行链式自动构建 [英] Chain automated builds in the same Docker Hub repository

问题描述

由于在Docker Hub上建立时间限制，我决定拆分 Dockerfile 耗时的自动构建到3个文件中。
这些子构建中的每一个都在Docker Hub的时间限制内完成。

Due to build time restrictions on Docker Hub, I decided to split the Dockerfile of a time-consuming automated build into 3 files. Each one of those "sub-builds" finishes within Docker Hub's time limits.

我现在在同一存储库中进行以下设置：

I have now the following setup within the same repository:

| branch | dockerfile         | tag    |
| ------ | ------------------ | ------ |
| master | /step-1.Dockerfile | step-1 |
| master | /step-2.Dockerfile | step-2 |
| master | /step-3.Dockerfile | step-3 |

图像按以下顺序相互构建：

The images build on each other in the following order:

• step-1.Dockerfile ： FROM ubuntu


• 步骤2.Dockerfile ：从我/复杂图像：步骤1


• step-3.Dockerfile ：从我/复杂图像：步骤2

• step-1.Dockerfile : FROM ubuntu
• step-2.Dockerfile : FROM me/complex-image:step-1
• step-3.Dockerfile : FROM me/complex-image:step-2

一个单独的Web应用程序触发使用 step-1 的构建Docker Hub提供的构建触发器 URL。 （向其中添加了 { docker_tag： step-1}'负载）但是，Docker Hub没有提供自动触发<$ c $的方法。 c>步骤2 ，然后步骤3 后缀。

A separate web app triggers the building of step-1 using the "build trigger" URL provided by Docker Hub. (to which the {"docker_tag": "step-1"}' payload is added) However, Docker Hub doesn't provide a way to automatically trigger step-2 then step-3 afterwords.

Q ：如何按顺序自动触发以下构建步骤？（即在 step-1之后触发 step-2 完成。然后，在 step-2 完成后触发步骤3 ）

Q: How can I automatically trigger the following build steps in their respective order ? (i.e. trigger step-2 after step-1 finishes. Then, trigger step-3 after step-2 finishes)

注意：我不想为每个 step-i 设置单独的存储库，然后使用Docker Hub的存储库链接。 我只想在同一存储库中链接标签。

NB: I don't want to setup separate repositories for each of step-i then link them using Docker Hub's "Repository Links." I just want to link tags in the same repo.

注意：到目前为止，我的解决方案是将Docker Hub Webhook附加到我制作的Web应用程序上。当 step-n 完成后，（即，使用包含标签的 step-n 的JSON调用我的Web应用程序的URL。 ），则网络应用使用生成触发器来触发 step-n + 1 。它按预期工作，但是，我想知道是否有一种更好的处理方式。

Note: Until now, my solution is to attach a Docker Hub Webhook to a web app that I've made. When step-n finishes, (i.e. calls my web app's URL with a JSON containing the tag name of step-n) the web app uses the "build trigger" to trigger step-n+1. It works as expected, however, I'm wondering whether there's a "better" way of doing things.

编辑： 根据Ken Cochrane的要求，这是初始 Dockerfile 以及构建脚本 。我只是想对 Cling 进行泊坞化。 （一个C ++解释器）它需要编译llvm，clang和cling。如您所料，根据机器的不同，它可能需要几个小时才能完成，并且Docker Hub最多只能仅进行2个小时的构建：）我后来添加的 sub build映像（仍在 develop 分支）分别构建整个事物的一部分。我不确定这里是否需要做进一步的优化。

As requested by Ken Cochrane Here are the initial Dockerfile as well as the "build script" that it uses. I was just trying to dockerize Cling. (a C++ interpreter) It needs to compile llvm, clang and cling. As you might expect, depending on the machine, it needs a few hours to do so, and Docker Hub allows "only" 2-hour builds at most :) The "sub build" images that I added later (still in the develop branch) build a part of the whole thing each. I'm not sure that there is any further optimization to be made here.

此外，为了测试各种想法（并避免等待小时的结果）我设置了具有类似结构的另一个存储库。 （唯一的区别是它的 Dockerfile 做的工作不多）

Also, in order to test various ideas (and avoid waiting h-hours for the result) I have setup another repo with a similar structure. (the only difference is that its Dockerfiles don't do as much work)

UPDATE 1 ：：在选项5 ：如预期的那样， curl 从 step-1.Dockerfile 中被忽略：

UPDATE 1: on Option 5: as expected, the curl from step-1.Dockerfile has been ignored:

Settings > Build Triggers > Last 10 Trigger Logs

| Date/Time                 | IP Address      | Status  | Status Description       | Request Body               | Build Request |
| ------------------------- | --------------- | ------- | ------------------------ | -------------------------- | ------------- |
| April 30th, 2016, 1:18 am | <my.ip.v4.addr> | ignored | Ignored, build throttle. | {u'docker_tag': u'step-2'} | null          |

此方法的另一个问题是，它要求我将构建触发器的（秘密）令牌放入 Dockerfile 供大家查看：)（希望Docker Hub可以选择使它无效并重新生成另一个文件）

Another problem with this approach is that it requires me to put the build trigger's (secret) token in the Dockerfile for everyone to see :) (hopefully, Docker Hub has an option to invalidate it and re-generate another one)

更新2：这是我当前的尝试：
它基本上是一个 Heroku 托管的应用，它具有 APScheduler 周期性的触发器，用于启动初始构建步骤，并使用 Flask webhook处理程序，用于传播构建。 （即，它具有构建标记的有序列表。每次Webhook调用它时，都会触发下一个构建步骤），我将在业余时间继续进行处理。 （欢迎提出建议：））

UPDATE 2: Here is my current attempt: It is basically a Heroku-hosted app that has an APScheduler periodic "trigger" that starts the initial build step, and a Flask webhook handler that "propagates" the build. (i.e. it has the ordered list of build tags. Each time it is called by the webhook, it triggers the next build step) I'll continue working on that on my spare time. (suggestions are welcome :) )

推荐答案

最近，对依赖链式构建的链接也有相同的要求，并使用Docker Cloud以这种方式实现自动化构建：

Recently had the same requirement to chain dependent builds, and achieved it this way using Docker Cloud automated builds:

• 为每个 Dockerfile 创建一个带有构建规则的存储库


• 对从属存储库中的所有构建规则禁用 Autobuild 选项。



• 在每个包含 Dockerfile 且具有以下依赖项的目录中的每个目录中添加名为 hooks\post_push 的shell脚本代码：

• Create a repository with build rules for each Dockerfile that needs to be built.
• Disable the Autobuild option for all build rules in dependent repositories.

• Add a shell script named hooks\post_push in each directory containing a Dockerfile that has dependents with the following code:

for url in $(echo $BUILD_TRIGGERS | sed "s/,/ /g"); do
  curl -X POST -H "Content-Type: application/json" --data "{ \"build\": true, \"source_name\": \"$SOURCE_BRANCH\" }" $url
done

对于每个具有依赖项的存储库，添加<将code>名为 BUILD_TRIGGERS 的构建环境变量设置为自动构建，并设置 Value 到每个相关的自动生成的生成触发URL的逗号分隔列表。

For each repository with dependents add a Build Environment Variable named BUILD_TRIGGERS to the automated build, and set the Value to a comma separated list of the build trigger URLs of each dependent automated build.

使用此设置进行推送到 root 源存储库后，一旦完成并按下 post_push 钩子，就会触发 root 映像的构建将被执行。在挂钩中，对每个从属存储库构建触发器进行POST，其中包含在请求正文中构建的分支或标记的名称。这将触发从属存储库的适当构建规则。

Using this setup a push to the root source repository will trigger a build of the root image, once it completes and is pushed the post_push hook will be executed. In the hook a POST is made to each dependent repositories build trigger, containing the name of the branch or tag being built in the requests body. This will cause the appropriate build rule of the dependent repository to be triggered.

这篇关于在同一Docker Hub存储库中进行链式自动构建的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！