从带有语义版本控制或Lerna Publish的CI/CD部署时，如何确保Master和Dev分支保持同步 [英] How to ensure Master and Dev branches are kept in sync when deploying from CI/CD with Semantic Versioning or Lerna Publish
问题描述
设置
我有几个gitlab存储库，其中的常规设置涉及一个 master 分支，一个 stage (预发布)分支和一个 dev 分支.
所有3个分支的推送权限均被禁用.
工作流将从 dev 分支派生，以获取所有修补程序，错误修复和功能.当您对发布感到满意时，您可以向 dev 提交合并请求.最终，当在 dev 内部准备好稳定的构建时；合并请求将提交给 stage 分支.最后，当您对预发行版本感到满意时，可以提交 master 分支的合并请求.
我配置了CI/CD，以便自动生成 CHANGELOG.md，从文件. master 和 stage 分支自动执行测试，构建和部署 stage 分支部署到UAT s3存储桶，而 master 分支部署到生产s3存储桶.
部署通过语义版本2.0.0 处理，该版本负责提高版本，生成变更日志并进行部署.
我有一个与上述相同的设置，除了它是一个monorepo，所以我使用 Lerna 来处理 {"conventionalCommits":true}的发布(部署).复制语义版本2.0.0 的行为.我在monorepo中使用独立版本控制.
语义版本2.0.0 和 Lerna 设置都强制 master 分支始终落后于或等于 stage 和 dev 分支；并且 stage 分支始终位于或等于 dev 分支，其效果类似于级联效果.
dev > = stage > = master
问题
Lerna Publish 和 Semantic Versioning 都在发布/部署时对文件进行了一些更改.其中一些更改包括更新 CHANGELOG.md 文件和在 package.json 文件内部增加版本.
Lerna和语义版本控制最终将这些更改推送到通过CI/CD运行的分支.
这意味着，如果我从 dev 合并到 stage ，则stage会通过语义版本控制或Lerna发布执行.这将导致 stage 分支位于 dev 分支之前，并且将导致 dev 分支中所有将来的派生与 stage 分支意味着下次我从 dev 合并到 stage 时，它将不会是简单的 fast-forward 合并，合并很可能会遇到冲突，这将阻止将来进行任何合并或使CI/CD失败.
我的解决方法
用于语义版本控制:
- 我已禁用推送功能，以便不再提交和推送新的更改文件(仍然创建和推送标签)
- 我创建了一个脚本，该脚本将
CHANGELOG.md文件转换为PDF并将其发送给我的团队电子邮件
之所以能奏效，是因为语义版本控制使用标签来确定更改的文件并决定如何更改版本.因此，尽管例如，回购中的版本保持恒定在 1.0.0 上，但是语义版本控制足够聪明，可以从最新的标记中增加版本，而不是从 package.json中的内容中获得最新的标记
不幸的是，这对于Lerna并不适用，Lerna仍然使用标签来确定更改的文件，但随后与 package.json 内部的版本发生冲突，这意味着不通过推送更新的 package.json 的新版本中，Lerna总是将我从 1.0.0 更改为 1.0.1 ， 1.1.0 或 2.0.0
所以我被Lerna困扰了.
问题
我应该如何设置CI/CD以避免出现此问题?我知道我的存储库结构很常见，尽管有无数Lerna和Semantic Versioning用户告诉我，我显然错过了一些东西，因为这不是一个广泛的问题，但是我还没有找到解决这个问题的人.
可能的解决方案
在我撰写此问题时，我想到，也许我只应该修改 dev 中的版本，然后从 stage 和 master .这样可以防止 stage 和 master 领先于 dev ，这是正确的方法吗?
在存储库中维护软件包版本信息不会扩展.尽管如此，所有可用的工具仍在继续尝试以使其正常工作.除了说那句话，我没有其他办法可以提供.发布数据应通过其他方式进行管理，而不是将其存储在源存储库中.我们在这里真正谈论的是异步过程，开发，构建和发布.这些系统的分布式性质意味着我们无法将回购视为文件共享，无法期望它们能够很好地扩展.
有关此主题，请参见我的其他人.我还没有时间对此做一个很好的记录.我要补充一点，Git标记是方便的里程碑标记，供开发人员找到要返回的正确git哈希值，以从中创建修订分支.提交消息用于更改日志，并且只是确定从哪个版本发布哪个版本的输入之一.
在多开发，多分支，分布式环境中工作的开发人员，不可能预测将来在某个随机点应用的适当语义版本.至少在没有对每个开发，分支和构建/发布环境的完全独裁控制的情况下.即使那样，他们也很难被迫使其发挥作用.当前工具所隐含的就是这种控制，但是在实践中，它永远无法扩展.
请考虑您的打包供稿服务可能包含全部或部分的发布历史记录.只要您只有一个Feed服务，就可以使用该服务来确定下一个发行版的版本底限.处理您的语义提交，查找与您的过程所针对的标签匹配的最新版本(每日，beta，RC，无或任何)，计算下一个适当的版本，在源代码中更新适当的版本字段，然后进行构建和测试.如果Feed服务不允许您在查询中包含隐藏或删除的版本，则必须使用自己的数据库.不要签入已修改的文件！这些字段应在您的存储库中清零，以有效地将本地开发版本标记为0.0.-dev或类似的内容.
可以自动发布预发行版本，但是发行版本中应该有一个人.如果上述所有步骤均成功，请将标签应用于刚刚成功构建的git哈希.
我梦想中的CI/CD系统，通过测试构建和单元测试运行对我的发行分支的每次提交，检测是否已修改现有测试用例(自动检测重大更改！)，分析提交消息，以了解表示有意损坏，并根据需要将所有这些信息提供给发布构建系统.我的发布构建系统生成一个-alpha.build.###并针对它运行所有验收测试.
如果没有已知的损坏，并且预定的目标是预发行版，则它将更新包含版本信息的文件，并在自动发布之前运行带有一次最终冒烟测试的增量构建.这是一些灰色区域.在没有人为干预的情况下，某些预发行目标不应包括破损，而对于其他目标，这是可以的.因此，我将拥有一组特殊的预发布目标，这些目标不允许自动发布破损，例如某些级别的狗食者或针对我内部长期运行的测试基础结构的比特.
如果这是一个未标记的发布目标，那么我希望它可以在我的测试的最后阶段构建并打包所有内容以供使用.在这里，自动化程序会检查软件包是否符合策略，确保可以正确解包，并在发布之前收集区域所有者，部门/部门负责人等的签字.在我们以实时系统为目标的情况下，它可能包括一些随机的测试部署.
以上所有内容实际上只是一个过度简化的描述.它掩盖了比其澄清的更多的东西，因为生产者和消费者在现实世界中的需求差异太大.
回到工具和控制上.DevOps世界中有许多人会告诉您，重点是围绕工具进行标准化.这使我想起此xkcd通讯.
Setup
I have several gitlab repos where the general setup involves having a master branch, a stage (pre-release) branch and a dev branch.
Push permissions are disabled for all 3 branches.
The workflow is to fork from the dev branch for any hot-fixes, bug fixes and features. When you are satisfied with the release you would submit a merge-request to dev. Eventually, when a stable build is ready inside dev; a merge-request would be submitted for the stage branch. Lastly, when you are satisfied with the pre-release you would submit a merge-request for the master branch.
I have CI/CD configured so that tests, builds and deployments are automatically executed from the master and stage branches with the automatic generation of CHANGELOG.md files. stage branch deploys to the UAT s3 Bucket and master deploys to the production s3 Bucket.
Deployment is handled through Semantic Versioning 2.0.0 which is responsible for bumping versions, generating changelogs and deploying.
I have a similar setup to the one just described above except it is a monorepo so I am using Lerna to handle the publishing (deploying) with {"conventionalCommits": true} to replicate Semantic Versioning 2.0.0's behaviour. I am using independent versioning inside the monorepo.
Both the Semantic Versioning 2.0.0 and the Lerna setup force the master branch to always be either behind or equal with the stage and dev branches; and the stage branch to always be behind or equal to the dev branch in kind of like a cascading effect.
dev >= stage >= master
The Problem
Both Lerna Publish and Semantic Versioning make several changes to the files when publishing/deploying. Some of these changes include updating the CHANGELOG.md file and bumping the version inside of the package.json file.
Both Lerna and Semantic Versioning eventually push these changes to the branch they are run from through the CI/CD.
What this means is that if I merge from dev to stage, stage would then have the bumped version numbers and new changelogs pushed into it through either the Semantic Versioning or Lerna Publish executions. This would cause the stage branch to be ahead of the dev branch and would cause all the future forks from the dev branch to detach from the stage branch meaning that the next time I merge from dev to stage it's not going to be a simple fast-forward merge like it's meant to be and the merge will most likely encounter conflicts which would prevent any future merges or may fail the CI/CD.
My Workaround
For Semantic Versioning:
- I have disabled the push feature so that the new, changed files are no longer committed and pushed (tags are still created and pushed)
- I have created a script that converts the
CHANGELOG.mdfile to a PDF and sends it to my teams email
This works out well because Semantic Versioning uses tags to determine the changed files and decide how to bump the version. So, although the version inside the repo remains constant at 1.0.0 for example, Semantic Versioning is smart enough to increment the version from the latest tag not from what's in the package.json
This unfortunately doesn't hold true for Lerna which still uses tags to determine changed files but then bumps from the version inside package.json which means that by not pushing the updated package.json with the new version, Lerna always bumps me from 1.0.0 to either 1.0.1, 1.1.0, or 2.0.0
So I am stumped with Lerna.
Question
How should I be setting up my CI/CD to avoid the problem? I know the structure of my repo is common, and I haven't found anyone addressing this issue despite the countless users of Lerna and Semantic Versioning which tells me that I have obviously missed something as it is not a wide-spread issue.
Possible Solution
As I was writing this question, it crossed my mind that maybe I should only bump versions in dev and then deploy from stage and master. This would prevent stage and master from ever being ahead of dev, would this be the right way to do it?
Maintaining package version information in the repo, does not scale. Still, all the tools out there keep trying to make it work. I have nothing to offer in the way of an alternative (yet), other than to say that; release data should be managed by other means, than storing it in the source repo. What we're really talking about here are asynchronous processes, dev, build and release. The distributed nature of these systems, implies that we cannot treat repos as file shares and expect them to scale well.
See my other rant on this topic. I haven't had time to do a good write-up on this yet. I would add that Git tags are meant to be handy milestone markers for devs to find the right git hash to go back to, to create a fix branch from. Commit messages are meant for change-logs and just one of the inputs into deciding what version to release from which build.
No dev, working in a multi-dev, multi-branch, distributed environment, can possibly predict the appropriate semantic version to apply at some random point in the future. At least not without having full dictatorial control of every dev, branch and build/release environment. Even then, they would be hard pressed to make it work. It's that control, that current tooling implies, but in practice, it never scales.
Consider that your package feed service likely has all, or a sufficient portion of your release history. As long you have only one feed service, you can use that to determine the version floor for your next release. Process your semantic commits, lookup the most recent version matching the tag your process is targeting (daily, beta, RC, none or whatever), calculate the next appropriate version, update appropriate version fields in source code, then build and test. If the feed service doesn't let you include hidden or deleted versions in your query, you'll have to use your own database. Do not check-in the modified files! Those fields should be zeroed in your repo, effectively marking local dev builds as 0.0.-dev or something along those lines.
Automatic publishing of prerelease versions is okay, but there should be a human in the loop for release versions. Iff all the above steps succeed, apply a tag to the git hash that you just successfully built from.
My dream CI/CD system, runs every commit to my release branch(es) through a test build and unit test runs, detects whether existing tests case were modified (auto-detect breaking changes!), analyzes the commit messages for indications of intentional breakage, and presents all of that information to the release build system on demand. My release build system produces a -alpha.build.### and runs all of the acceptance tests against it.
If there is no known breakage and the intended target is a prerelease, it then updates the files containing version information and runs an incremental build with one final smoke test prior to automatic publication. Here's where there's some gray areas. Some prerelease targets should not include breakage, without human intervention, and for others it's okay. So I would have a special set of prerelease targets that do not allow automated publication of breakage, such as certain levels of dog-fooders, or bits targeted at my internal long running test infrastructure.
If it's an untagged release target, then I would prefer it to build and package everything for consumption in my final stage of testing. This is where the automation verifies the package for conformance to policies, ensures that it can be unpacked correctly, and gathers sign-offs from area owners, department/division heads, etc, prior to publication. It might include some randomized test deployments, in cases where we're targeting a live system.
And all of the above is just an over-simplified description really. It glosses over more than it clarifies, because there's just too much variation in the real-world needs of producers and consumers.
Circling back to tools and control. There are many in the DevOps world who will tell you that the main point is to standardize around tooling. Which remind me of this xkcd commic.
这篇关于从带有语义版本控制或Lerna Publish的CI/CD部署时，如何确保Master和Dev分支保持同步的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
