如何使用多个 docker 设置 Gitlab CI E2E 测试 [英] How to setup Gitlab CI E2E tests using Multiple dockers

问题描述

我对使用 Gitlab CI 的自动化测试有点迷茫.我希望我能解释我的问题，以便有人可以帮助我.我会先尝试解释情况，然后我会尝试提出一个问题(这比听起来更难)

I am a bit lost with the automated testing using Gitlab CI. I hope I can explain my problem so somebody can help me. I'll try to explain the situation first, after which I'll try to ask a question (which is harder than it sounds)

1. 使用 Jest 单元测试和 Cypress e2e 测试响应前端
2. Django API 服务器 1，包括 Postgres 数据库和测试
3. 带有 MongoDB 数据库的 Django API 服务器 2(与其他 API 通信

Gitlab

对于 2 个 API，有一个 Docker 和一个 docker-compose 文件.这些工作正常并且设置正确.

Gitlab

For the 2 API's, there is a Docker and a docker-compose file. These work fine and are set up correctly.

我们将 GitLab 用于 CI/CD，我们按此顺序分为以下阶段:

We are using GitLab for the CI/CD, there we have the following stages in this order:

1. build: 其中 docker 用于 1、2 和3 个单独构建并推送到私有注册表
2. 测试:单元测试和端到端测试(应该)在哪里运行
3. 发布: docker 镜像的发布位置
4. 部署: docker 镜像的部署位置

1. build: where dockers for 1, 2 & 3 are build separate and pushed to private-registry
2. Test: Where the unit testing and e2e test (should) run
3. Release: where the docker images are released
4. Deploy: Where the docker images are deployed

目标

我想设置 GitLab CI 以便它运行 cypress 测试.但是为此，需要所有构建泊坞窗.目前，我无法在执行端到端测试时同时使用所有 docker.

Goal

I want to set up the GitLab CI such that it runs the cypress tests. But for this, all build dockers are needed. Currently, I am not able to use all dockers together when performing the end-to-end tests.

我真的不知道我将如何实现这一目标.

I don't really get how I would achieve this.

• 我可以将在构建阶段构建的 docker 用于我的 e2e 测试吗?有人可以举例说明如何实现这一点吗?(通过将构建 docker 容器作为服务运行?)
• 我是否需要一个包含所有 docker 和数据库的 Docker-compose 文件?
• 我还需要dind吗?

我希望有人可以就如何实现这一目标给我一些建议.一个例子会更好，但我不知道是否有人愿意这样做.

I hope somebody can give me some advice on how to achieve this. An example would be even better but I don't know if somebody would want to do that.

感谢您抽出时间阅读！

build-api:
  image: docker:19
  stage: build
  services:
    - docker:19-dind
  script:
    cd api
    docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    docker pull $IMAGE_TAG_API:latest || true
    docker build -f ./Dockerfile --cache-from $IMAGE_TAG_API:latest --tag $IMAGE_TAG_API:$CI_COMMIT_SHA .
    docker push $IMAGE_TAG_API:$CI_COMMIT_SHA

test-api:
  image: docker:19
  stage: test
  services:
    - postgres:12.2-alpine
    - docker:19-dind
  variables:
    DB_NAME: project_ci_test
    POSTGRES_HOST_AUTH_METHOD: trust
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker pull $IMAGE_TAG_API:$CI_COMMIT_SHA
    - docker run $IMAGE_TAG_API:$CI_COMMIT_SHA sh -c "python manage.py test"
  after_script:
    - echo "Pytest tests complete"
  coverage: "/TOTAL.+ ([0-9]{1,3}%)/"

release-api-staging:
  image: docker:19
  stage: release
  services:
    - docker:19-dind
  only:
    refs: [ master ]
    changes: [ ".gitlab-ci.yml", "api/**/*" ]
  environment:
    name: staging
  script:
    - docker pull $IMAGE_TAG_API:$CI_COMMIT_SHA
    - docker tag $IMAGE_TAG_API:$CI_COMMIT_SHA $IMAGE_TAG_API:latest
    - docker push $IMAGE_TAG_API:latest

推荐答案

答案有点晚了，但我还是会尝试 为其他遇到相同问题的开发人员简要解释该方法.我还创建了一个 示例项目，在 GitLab 中包含 3 个微服务，其中服务器 A 运行端到端测试和依赖于服务器 B 和服务器 C.

The answer is a bit late, but still i'll try to explain the approach briefly for other developers with same issues. I also created an example project, contain 3 microservices in GitLab, where Server A runs end-to-end tests and is dependend on Server B and Server C.

在端到端测试全栈应用程序时，您必须:

When e2e test full-stack applications you have to either:

• 模拟微服务的所有响应
• 针对已部署的环境进行测试；
• 或在管道中临时启动环境

正如您所指出的，您希望在管道中临时启动环境.应采取以下步骤:

As you noted, you want to spin-up the environment temporary in the pipeline. The following steps should be taken:

1. 将所有后端部署为 GitLab 私有注册表中的 docker 镜像；
2. 在管道中的 1 个作业中模仿您的 docker-compose.yml 服务；
3. 将点连接在一起.

在 GitLab 私有注册表中将后端部署为 docker 镜像

首先，您必须在 GitLab 的私有注册表中发布您的 docker 镜像.您这样做是因为您现在可以在另一项工作中重复使用这些图像.对于这种方法，您需要 docker:dind.在 gitlab 上发布到私有注册表的简单示例作业如下所示:

Deploy backends as docker images in GitLab private registry

First you have to publish your docker images in the private registry of GitLab. You do this, because you now can reuse those images in another job. For this approach you need docker:dind. A simple example job to publish to a private registry on gitlab looks like:

before_script:
  - echo -n $CI_JOB_TOKEN | docker login -u gitlab-ci-token --password-stdin $CI_REGISTRY

publish:image:docker:
  stage: publish
  image: docker
  services:
    - name: docker:dind
      alias: docker
  variables:
    CI_DOCKER_NAME: ${CI_REGISTRY_IMAGE}/my-docker-image
  script:
    - docker pull $CI_REGISTRY_IMAGE || true
    - docker build --pull --cache-from $CI_REGISTRY_IMAGE --tag $CI_DOCKER_NAME --file Dockerfile .
    - docker push $CI_DOCKER_NAME
  only:
    - master

要查看真实示例，我有一个公开可用的示例项目.

To see a real-world example, I have an example project that is public available.

一旦您将所有后端都docker化并在私有注册表上发布了图像，您就可以开始使用 GitLab 作业来模拟您的 docker-compose.yml.一个基本的例子:

Once you dockerized all backends and published the images on a private registry, you can start to mimic your docker-compose.yml with a GitLab job. A basic example:

test:e2e:
   image: ubuntu:20.04
   stage: test
   services:
      - name: postgres:12-alpine
        alias: postgress
      - name: mongo
        alias: mongo
      # my backend image
      - name: registry.gitlab.com/[MY_GROUP]/my-docker-image
        alias: server
   script:
      - curl http://server:3000 # expecting server exposes on port 3000, this should work
      - curl http://mongo:270117 # should work
      - curl http://postgress:5432 # should work!

运行测试

现在所有东西都在 GitLab 的单个作业中运行，您可以简单地以分离模式启动前端并运行 cypress 来测试它.示例:

Run the tests

Now everything is running in a single job in GitLab, you can simply start your front-end in detached mode and run cypress to test it. Example:

 script:
   - npm run start & # start in detached mode
   - wait-on http://localhost:8080 # see: https://www.npmjs.com/package/wait-on
   - cypress run # make sure cypress is available as well

结论

您的 docker-compose.yml 不打算在管道中运行.使用 GitLab 服务来模仿它.Dockerize 所有后端并将它们存储在 GitLab 的私有注册表中.启动管道中的所有服务并运行测试.

Conclusion

Your docker-compose.yml is not meant to run in a pipeline. Mimic it instead using GitLab services. Dockerize all backends and store them in GitLab's private registry. Spin up all services in your pipeline and run your tests.

这篇关于如何使用多个 docker 设置 Gitlab CI E2E 测试的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！