在CI / CD中，如何管理前端和后端之间的依赖关系？ [英] In CI/CD how to manage dependency between frontend and backend?

问题描述

我将描述我的设置以使问题不太抽象，但问题似乎并非我的情况所独有。

上下文

我们在存储库中分别具有Python-Django后端和VueJS前端，其中配置了Gitlab-CI，并使用Portainer（使用堆栈）进行了部署。
每个存储库生产分支中的提交都遵循以下路径：

1. commit


2. gitlab-ci管道：
   
   
   
   
   1. 构建docker映像
   
   
   2. 测试映像（前端已针对部署的后端进行了测试）
   
   
   3. 将映像标记为production：latest
   
   
   4. 将映像推送回gitlab注册表
   
   
   5. webhook相应的服务（前端/后端）以更新已部署的映像
   
   



3. tainer：
   
   
   
   
   1. 拉出图像
   
   
   2. 部署
   
   

问题

部署同步

想象一下我们在前端和后端都做了重大更改将与以前的版本不兼容。因此，必须同时部署新版本。

在我们当前的设置中，我们必须首先部署后端（将会破坏已部署的前端），然后再部署新的前端，

测试的分支依赖性

有时我们开发分支前端的feature-1，必须针对后端的分支Feature-1进行测试。

在我们当前的设置中，前端中的所有提交都针对已部署的对象进行了测试后端（为避免在CI中复制后端，仅使用生产API地址），在这种情况下会导致错误的测试结果。

后端集成测试

对后端的提交会破坏前端。

当前后端尚未针对后端进行测试

可能的解决方案

用于部署同步问题我考虑过创建另一个仅具有一个文件的存储库，该文件指定应部署的前端和后端的版本。在此存储库中的提交将导致Portanier的服务网络钩子都被卷曲以进行更新（后端和前端）。这不能保证同时进行更新（在Portainer中可能会失败，并且不会回滚），但是比当前设置要好。

我不是确定应该在此处指定版本的方法：提交哈希，git标签，分支，docker映像版本...最后一个可以避免必须重建和测试映像，但是我认为映像名称和版本在Portainer的堆栈定义中是固定的，而且不容易自动更新。

对于分支依赖测试，我考虑过要在每个存储库（前端和后端）中都有一个文件，指定从后端/前端的哪个分支进行测试。但是，每个存储库的CI必须复制整个部署环境（例如，运行新的后端和前端以测试每个前端提交）。这也将允许后端集成测试。由于我们使用的是Docker，这并不是很复杂，但是每个CI管道都需要花费额外的时间...而且，当第一个存储库（前端或后端）被提交时，它将在另一个存储库中引用一个尚不存在的分支存储库，然后失败...

这些解决方案对我来说似乎很尴尬，特别是如果这些是Docker CI / CD常见的问题。当我们向组合中添加更多存储库时，它甚至变得更糟。

替代品？

感谢注意！

（ edit：出于好奇，我当前的设置基于此文章）

解决方案

部署同步

想象一下我们在前端和后端都进行了重大更改，并且两者都将变得不兼容与以前的版本。因此，新版本必须同时部署。

在我们当前的设置中，我们必须首先部署后端（将会破坏已部署的前端），然后部署新的前端，以固定生产，但是带有向下的

我不是Portainer用户，但也许您可以依靠某些 docker-compose.yml 文件左右，同时收集后端和前端的版本？在这种情况下，它们可以同时更新...

实际上是根据 portainer / portainer＃1963 和此文档页面，portainer似乎同时支持docker-compose和swarm堆栈。

此外，docker swarm提供了一些功能来执行服务升级而无需停机，如此博客，但我不知道可以在portainer中配置多大程度。

可能的解决方案

I我不确定在这里指定版本时应该使用什么：提交哈希，git标签，分支，docker映像版本...最后一个可以避免必须重建和测试映像，但是我认为图像名称和版本在Portainer的堆栈定义中是固定的，并且不容易自动更新。

虽然提交哈希是精确的标识符，但它们可能是无法方便地识别不兼容的版本。因此，您可能需要使用Git后端存储库上的标签（和/或分支）来依赖语义版本控制。 p>

然后，您可以相应地标记相应的Docker映像，并在需要时引入一些同义标记。例如，假设后端已发布，版本为 1.0.0、1.0.1、1.1.0、1.1.1、1.2.0、1.2.1、1.2.2 ，标准做法是像这样标记Docker映像：

• project / backend：2.0.2 = 项目/后端：2.0 = 项目/后端：2


• 项目/后端：2.0.1


• 项目/后端：2.0.0


• 项目/后端：1.1.1 = 项目/后端：1.1 = 项目/后端：1


• 项目/后端：1.1.0


• 项目/后端：1.0.1 = 项目/后端：1.0


• 项目/后端：1.0.0

（如果需要的话，删除旧图像）

后端集成测试

当前未对后端进行测试

好的，但是我想您的方法相当合理ard（前端取决于后端，而不是后端）。

无论如何，我记得即使被测系统是前端，也可能值得实施单元测试（与集成测试相比，这种方法的开发和运行成本较低），以便在触发必要的集成测试之前，管道的第一阶段可以快速运行这些单元测试。

测试的分支依赖性

在我们当前的设置中，前端中的所有提交都针对已部署的后端进行了测试（为避免在CI中复制后端，只有生产API地址

这可能不够灵活：通常，CI / CD假定集成测试是使用专用后端实例（ dev服务器或预生产服务器服务器），并且如果所有集成测试和系统测试通过，则将映像部署到 prod服务器上。服务器（并受监视等）

我从您的帖子中看到您正在使用GitLab CI，其中包含一些本地Docker支持，因此也许可以轻松实现。

一些提示：

• 假设后端已在非向后兼容版本中进行了修改，并且相应的Docker映像在注册表中可用（例如，GitLab CI的映像） 。然后，您可以在前端配置中更改该映像的规范（例如，将 project / backend：1 替换为 project / backend：2





• 您的后端可能实现为REST Web服务，在这种情况下您可能还想在URL中添加版本前缀，以便从 project / backend：1 切换到 project / backend：2 （具有不兼容的更改），如果需要，可以同时将两个版本部署到URL https://example.com/api/v1/…和 https：//example.com/api/v2 / ...

  
  

此外，除了只有两个带有CI / CD的存储库（后端经过测试，并且前端针对后端的相关版本进行测试）的解决方案之外，您最初建议的解决方案可能是考虑过：

对于我考虑创建的部署同步问题另一个存储库，只有一个文件，该文件指定应部署的前端和后端的版本。对该存储库中的提交将导致Portanier的两个服务Webhooks被卷曲或被卷入。用于更新（后端和前端）。这不能保证同时进行更新（在Portainer中可能会失败，并且不会回滚），但是比当前设置要好。

您可以略微修改此方法，以避免出现这样的部署失败：您可以在第三个存储库中添加一些配置项设置，该配置项仅包含 docker-compose.yml 文件，并将集成测试从前端CI移至该组合。 CI ...

（仅供参考，此方法类似于此 DigitalOcean教程，其中通过一些 docker-compose.test.yml 文件实现了集成测试。）

I'll describe my setup to make the problems less abstract, but they don't seem specific to my case.

Context

We have Python-Django backend and a VueJS frontend, each in a repository, with Gitlab-CI configured and deploy using Portainer (using a stack). A commit in the production branch of each repository follows this path:

1. commit
2. gitlab-ci pipeline:
   1. build docker image
   2. test image (frontend is tested against deployed backend)
   3. tag image as production:latest
   4. push image back to gitlab registry
   5. webhook the corresponding service (frontend/backend) in portainer to update the deployed image

3. portainer:
   1. pull image
   2. deploy

Problems

Deployment synchronization

Imagine we're doing a major change in both frontend and backend, and both will become incompatible with previous versions. So the new versions must be deployed simultaneously.

In our current setup we have to first deploy the backend (what will break the deployed frontend) and then deploy the new frontend, fixing production, but with a "down" period.

Branch dependency for tests

Sometimes when we develop branch feature-1 in the frontend, it must be tested against branch feature-1 from the backend.

In our current setup all the commits in the frontend are tested against the deployed backend (to avoid replicating the backend in CI, only the production API address is used), resulting in false tests results in such cases.

Backend integration tests

When a commit is done to the backend, it can break the frontend.

Currently the backend isn't tested against the frontend (only the other way).

Possible Solutions

For the deployment synchronization problem I thought about creating another repository that would have only one file specifying the versions for frontend and backend that should be deployed. A commit in this repository would result in both Portanier' services webhooks being "curled" for update (backend and frontend). This doesn't guarantee the simultaneous update (one may fail in Portainer and there would be no rollback), but it would be better than current setup.

I'm not sure about what should be used to specify versions here: commit hash, git tag, branch, docker image version... The last maybe avoids having to rebuild and test images, but I think images name and versions are fixed in Portainer' stacks definition, and not easy to update automatically.

For the branch dependency tests I thought about having a file in each repository (frontend and backend) specifying which branch from the backend/frontend to test against. But the CI for each repository would have to replicate the whole deploy environment (running a new backend and frontend to test each frontend commit, for example). This would also allow backend integration tests. Since we're using Docker, that's not very complicated, but will take extra time for each CI pipeline... Also, when the first repository (frontend or backend) is committed, it'll reference a still non existent branch in the other repository, and fail...

These solutions seem awkward to me, specially if these are problems common to CI/CD with Docker. And it can get even uglier when we add more repositories to the mix.

Alternatives?

Thanks for the attention!

(edit: for curious, my current setup was based on this article)

解决方案

Deployment synchronization

Imagine we're doing a major change in both frontend and backend, and both will become incompatible with previous versions. So the new versions must be deployed simultaneously.

In our current setup we have to first deploy the backend (what will break the deployed frontend) and then deploy the new frontend, fixing production, but with a "down" period.

I'm not a portainer user, but maybe you could rely on some docker-compose.yml file or so, gathering both the version of the backend and the frontend? in this case they could be updated at the same time…

Indeed according to portainer/portainer#1963 and this doc page, portainer seems to support both docker-compose and swarm stacks.

Also, docker swarm provides some features to perform service upgrade without downtime, as documented in this blog, but I don't know to what extent this can be configured in portainer.

Possible Solutions

I'm not sure about what should be used to specify versions here: commit hash, git tag, branch, docker image version... The last maybe avoids having to rebuild and test images, but I think images name and versions are fixed in Portainer' stacks definition, and not easy to update automatically.

While commit hashes are precise identifiers, they are probably not convenient enough to identify incompatible versions. So you may want to rely on semantic versioning using tags (and/or branches) on your Git backend repo.

Then, you may tag the corresponding Docker images accordingly, introducing some synonymous tags if need be. For example, assuming the backend has been released with versions 1.0.0, 1.0.1, 1.1.0, 1.1.1, 1.2.0, 1.2.1, 1.2.2, a standard practice consists in tagging the Docker images like this:

• project/backend:2.0.2 = project/backend:2.0 = project/backend:2
• project/backend:2.0.1
• project/backend:2.0.0
• project/backend:1.1.1 = project/backend:1.1 = project/backend:1
• project/backend:1.1.0
• project/backend:1.0.1 = project/backend:1.0
• project/backend:1.0.0

(removing old images if need be)

Backend integration tests

Currently the backend isn't tested against the frontend (only the other way).

OK but I guess your approach is fairly standard (the frontend depends on the backend, not the other way around).

Anyway, I recall that even if the system under test is a front-end, it may be worth it to implement unit tests (which are less costly to develop and run than integration tests) so that a first stage in the pipeline quickly runs these unit tests, before triggering the necessary integration tests.

Branch dependency for tests

In our current setup all the commits in the frontend are tested against the deployed backend (to avoid replicating the backend in CI, only the production API address is used), resulting in false tests results in such cases.

This may be not flexible enough: in general, CI/CD assumes the integration tests are run using a dedicated backend instance ("dev" server or "pre-prod" server), and if all integration tests and system tests pass, the image is deployed to the "prod" server (and monitored, etc.)

I see from your post that you are using GitLab CI, which has some native Docker support, so maybe this could be implemented easily.

A couple of hints:

• Assume the backend has been modified in a non-backward compatible version, and the corresponding Docker image is available in a registry (e.g. that of GitLab CI). Then you could just change the specification of that image in the frontend configuration (e.g., replacing project/backend:1 with project/backend:2 or so in the GitLab CI conffile).

• Your backend is probably implemented as a REST Web Service, in which case you might also want to add a version prefix in your URL, so that when you switch from project/backend:1 to project/backend:2 (with incompatible changes), both versions could be deployed at the same time if need be, to the URLs https://example.com/api/v1/… and https://example.com/api/v2/…

Also, beyond the solution to have only two repos with CI/CD (backend tested apart, and frontend tested against the relevant version of the backend), the solution you suggested in the first place may also be considered:

For the deployment synchronization problem I thought about creating another repository that would have only one file specifying the versions for frontend and backend that should be deployed. A commit in this repository would result in both Portanier' services webhooks being "curled" for update (backend and frontend). This doesn't guarantee the simultaneous update (one may fail in Portainer and there would be no rollback), but it would be better than current setup.

You could slightly modify this approach to avoid one such deployment failure: you could add some CI setup to that third repo, that would only contain a docker-compose.yml file or so, and move the integration tests from the frontend CI to that "compose" CI…

(FYI this approach is similar to the one suggested in this DigitalOcean tutorial, where the integration testing is achieved thanks to some docker-compose.test.yml file.)

这篇关于在CI / CD中，如何管理前端和后端之间的依赖关系？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！