如何在同一实例中运行Gitlab CI作业 [英] How to run Gitlab CI jobs in the same instance

问题描述

我已经在 AWS 竞价型实例上自动缩放了gitlab-runner.而且效果很好.

I have autoscaled the gitlab-runner on AWS spot instances. And it works fine.

运行作业时出现问题.下面是我的 .gitlab-ci.yml ，它分为两个阶段.

And I have an issue when running the jobs. Below is my .gitlab-ci.yml and it has two stages.

stages:
 - build
 - dev1:build

build:
 stage: build
 script: 
  - docker build --rm -t broker-connect-dev1-${CI_COMMIT_SHORT_SHA} -f BrokerConnect/Dockerfile .
 only:
  - dev1/release
 tags:
  - itela-spot-runner     

build-dev1:
 stage: dev1:build
 script: 
  - docker tag broker-connect-dev1-${CI_COMMIT_SHORT_SHA}:latest 19950818/broker-connect:${DEV1_TAG} 
 only:
  - dev1/release
 tags:
  - itela-spot-runner  

问题来了，因为我使用竞价型实例运行作业，有时 build 阶段发生在一个竞价型实例中，而 dev1:build 阶段发生在另一个现场实例.发生这种情况时， dev1:build 失败，因为它找不到图像 broker-connect-dev1-$ {CI_COMMIT_SHORT_SHA} ，因为它是在单独的竞价型实例中构建的.在gitlab或gitlab-runner中，是否有一种方法可以控制此行为，并在同一个竞价型实例中运行这两个作业 build 和 dev1:build ?

And here comes the problem, since I am using spot instances to run the jobs sometimes the build stage happens in one spot instance and the dev1:build stage happens in another spot instance. When this happens dev1:build fails as it cannot find the image broker-connect-dev1-${CI_COMMIT_SHORT_SHA} because it has been built in a separate spot instance. In gitlab, or in gitlab-runner, is there a way to control this behavior and run these two jobs build and dev1:build in the same spot instance?

推荐答案

控制哪些作业在哪个跑步者上运行的最佳方法是使用标签.您可以为跑步者添加诸如 builds-images 之类的标签，然后在构建图像的任何作业上，或者需要使用上一步构建的图像时，都使用相同的标签.

The best way to control which jobs run on which runners is by using tags. You could tag a runner something like builds-images, then on any jobs that build images, or need to use images built by a previous step, use the same tag.

例如:

stages:
 - build
 - dev1:build

build:
 stage: build
 script: 
  - docker build --rm -t broker-connect-dev1-${CI_COMMIT_SHORT_SHA} -f BrokerConnect/Dockerfile .
 only:
  - dev1/release
 tags:
  - itela-spot-runner
  - builds-images   

build-dev1:
 stage: dev1:build
 script: 
  - docker tag broker-connect-dev1-${CI_COMMIT_SHORT_SHA}:latest 19950818/broker-connect:${DEV1_TAG} 
 only:
  - dev1/release
 tags:
  - itela-spot-runner
  - builds-images

现在，您只需要一个带有 builds-images 标签的跑步者即可.如果您使用的是gitlab.com或是自托管的，并且至少具有Gitlab版本13.2，则可以在项目的运行者"页面中编辑运行者的详细信息(详细信息在这里:

Now you just need to have a runner (or runners) tagged with builds-images. If you're using gitlab.com or are self-hosted and have at least Gitlab version 13.2, you can edit a runner's details in the Runners page for a project (details here: https://docs.gitlab.com/ee/ci/runners/#view-and-manage-group-runners). Otherwise, tags can be set while registering a runner. For your use case, without further changing your .gitlab-ci.yml file, I'd only tag one runner.

另一种选择是将构建的映像推送到任一Docker中心( https://docs.docker.com/docker-hub/)，Gitlab的注册表( https://docs.gitlab.com/ee/user/packages/container_registry/)或其他可以支持docker映像的注册表( https://aws.amazon.com/ecr/).然后在需要该映像的所有作业上，将其从注册表中拉出并使用.

The other option is to push the built image to either docker hub (https://docs.docker.com/docker-hub/), Gitlab's registry (https://docs.gitlab.com/ee/user/packages/container_registry/), or another registry that can support docker images (https://aws.amazon.com/ecr/). Then on any jobs that need the image, pull it down from the registry and use it.

以您的示例为例:

stages:
 - build
 - dev1:build

build:
 stage: build
 before_script:
   - docker login [registry_url] #...
 script: 
  - docker build --rm -t broker-connect-dev1-${CI_COMMIT_SHORT_SHA} -f BrokerConnect/Dockerfile .
  - docker push broker-connect-dev1-${CI_COMMIT_SHORT_SHA}
 only:
  - dev1/release
 tags:
  - itela-spot-runner     

build-dev1:
 stage: dev1:build
 before_script:
   - docker login [registry_url] #...
 script: 
  - docker pull broker-connect-dev1-${CI_COMMIT_SHORT_SHA}
  - docker tag broker-connect-dev1-${CI_COMMIT_SHORT_SHA}:latest 19950818/broker-connect:${DEV1_TAG} 
 only:
  - dev1/release
 tags:
  - itela-spot-runner

这篇关于如何在同一实例中运行Gitlab CI作业的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！