在源代码和预编译二进制文件之间切换 [英] Switch between source code and precompiled binaries
问题描述
我们的应用程序中有大量的库.库是用 c++ 或 c# 编写的.(平台:.net 框架，windows，64 位)将所有内容编译为源代码需要很多时间.我们正在考虑切换到预构建的二进制文件，但我们仍然希望保留返回源代码的可能性.作为我们使用 git 的版本控制系统，构建可以由 azure devops 完成.可以根据需要设置任何自定义服务器.
有哪些现成的工具可用于包管理，以及在源代码和预构建二进制文件之间轻松切换的可能性?(如果工具与两种编程语言不兼容 - 可以只为一种语言指定工具集.)如果这样的工具不存在，你会自己推荐什么 - 使用什么样的包装，写什么样的脚本上面那个?
是否可以识别 Api/abi 中断?
这个问题有多个轴 - 源代码构建、包的安装、外部存储库、api/abi 中断、构建系统本身.
首先是您的要求 - 您是否只对包装和/或安装感兴趣.
对于包装本身，可以使用例如以下包装系统:nuget、conan、vcpkg、choco.
很可能您不仅会对发布包本身感兴趣，还会对它们的调试符号感兴趣.
对于 azure devops/调试符号发布文档可以从以下链接中找到，例如:
https://azure.microsoft.com/en-us/blog/deep-dive-into-azure-artifacts/https://docs.microsoft.com/en-我们/nuget/create-packages/symbol-packages-snupkg
Conan(c++包)目前不支持符号发布:https://github.com/conan-io/conan/issues/4047
要获得源代码分发，可以使用 git 本身.Git 子模块确实允许创建外部 git 存储库引用，但是一旦添加了外部子模块，您需要使用双重提交来更改所有内容 - 一次 git 提交到子存储库，一次 git 提交从主存储库更新引用的 git 哈希.
确实存在一些解决此问题的方法，例如:没有子模块的嵌套 git:
=>
http://debuggable.com/posts/git-fake-submodules:4b563ee4-f3cc-4061-967e-0e48cbdd56cb
如果您使用预构建的二进制文件而不是源代码构建主存储库 - 在理想的解决方案中，您不需要外部 git 存储库，这也意味着不需要 git clone/检出外部存储库.
那么也可以有多合一解决方案或多个解决方案，每个解决方案单独编译.
确实更容易拥有多合一解决方案，因为它减少了依赖解决方案编译的麻烦.
为了实现完全动态的解决方案生成，例如可以使用 cmake - 它可以根据从 cmake 使用的预配置选项生成解决方案.(参见 cmake 的 "option").
cmake 对于 C++ 确实是很好的替代品(支持预编译头文件，加快统一构建速度)，但对于 C# 可能不是那么简单(很难找到如何正确配置它的必要参数).
目前(5.2020)还没有找到比 cmake 更好的替代品，但是有多种工具和生态系统与 cmake 并行发展，因此需要观察未来可能带来什么.
然后是关于包装.choco 似乎是 nuget 的一个子集——它扩展了 nuspec，它有自己的支持软件更新的 xml 标签.(参考网页:https://chocolatey.org/、https://docs.microsoft.com/en-us/nuget/reference/nuspec)
如果您想发布 nuget 包甚至安装包，可以使用 nuget 存储库(无需任何额外费用).
如果不足以满足您的需求，也可以将 nuget 服务器升级到 choco 服务器，但可能需要一些费用 - 请参阅巧克力网页.
在任何级别上都无法观察到 Api/abi 中断，直到实际发生编译/链接错误或应用程序在运行时崩溃 - 唯一可能的选择 - 是控制包装本身的版本控制 - 所以主包 nuget(或巧克力)版本会需要更高版本的依赖 nuget(或 choco)版本.
如果主存储库与子存储库由同一个人开发 - 那么 api/abi 中断可以由开发人员自己完成，但是如果两个 git 彼此独立并由不同的团队开发 - 那么 api/abi 中断可能发生在任何时间点.
确保存储库一致的最佳方法是对主存储库进行单元测试，这将检查没有发生 api/abi 中断.
理论上，如果发生 api/abi 中断 - 构建系统本身可以继续使用两个构建替代方案 -
- 关注主存储库和最后一个工作子存储库
- 关注子仓库而不构建主仓库
可以通过使用分支来应用相同的机制，例如
- "主 git: master 分支" + "子 git: release/1.0 分支"
- 子git:主分支"
这种机制可能不仅需要观察两个存储库，还需要在长期失败的情况下切换分支.(例如，子 git 的开发团队不关心主 git 中发生的构建中断)
怀疑不存在用于此目的的任何现成工具(如果出现此类工具，请留下评论)，因为这可能需要重新连接来自两个可能不同的构建链的工具，这些构建链来自可能不同的 git 存储库，来自可能不同的组织.
最后但并非最不重要的 - 如果您希望您的应用程序能够进行软件更新 - 那么 nuget/choco 分发下载可以与额外的 nuget/choco 服务器一起使用.
这里是 nuget 包下载部分的好例子:https://github.com/mwrock/NugetDownloadFeed
安装包构建是一个更复杂的问题 - 例如参见以下链接:
(根据我的意见，按最佳顺序排列)
We have huge amount of libraries in our application. Libraries are written either in c++ or c#. (Platform: .net framework, windows, 64 bit) Compiling everything as source code takes a lot of time. We were thinking about switching to prebuilt binaries, but we still would like to leave a possibility to return back to source code. As a version control system we use git, builds can be done by azure devops. Can set up any custom server as seen necessary.
What kind ready made tools there exists for package management, and possibility to easy switch between source code and pre-built binaries? (If tools are not compatible with both programming languages - its ok to specify toolset only for one language.) If such tools does not exists, what you would recommend by yourself - what kind of packaging to use, what kind of scripts to write on top of that one ?
Is it possible to identify Api/abi breaks ?
There are multiple axis on this problem - source code building, installation of package, external repositories, api/abi breaks, build system itself.
First of all your requirements - are you interested only in packaging and/or also installation.
For packaging itself it's possible to use for example following packaging systems:
nuget, conan, vcpkg, choco.
Most probably you will be also interested not only publishing the packages itself, but also debug symbols for them.
For azure devops / debug symbols publishing documentation can be found from following links for example:
https://azure.microsoft.com/en-us/blog/deep-dive-into-azure-artifacts/ https://docs.microsoft.com/en-us/nuget/create-packages/symbol-packages-snupkg
Conan (c++ packaging) at the moment does not support symbols publishing: https://github.com/conan-io/conan/issues/4047
To get source code distribution it's possible to use git itself. Git submodules does allow to create external git repository reference, but once you have added external submodule, you need to use double commit to change everything - one git commit to sub repository, one git commit to update referenced git hash from master repository.
There indeed exists some walkarounds to this problems like: nested git without submodules:
Nested git repositories without submodules?
=>
http://debuggable.com/posts/git-fake-submodules:4b563ee4-f3cc-4061-967e-0e48cbdd56cb
If you build however main repository with prebuilt binaries instead of source codes - in ideal solution you don't need that external git repository, meaning also that no git clone / checkout of external repository is needed.
Then it's also possible to have either all-in-one solution or multiple solutions, each compiled separately.
Easier indeed to have all-in-one solution, because it's less hassle with dependent solutions compilation.
To achieve fully dynamic solution generation, it's possible for example to use cmake - it can generate solution depending on preconfigured options used from cmake. (See cmake's "option").
cmake is good alternative indeed for C++ (supports precompiled header, unity building speed up), but maybe not so straightforward for C# (difficult to find necessary parameters how to configure it correctly).
At the moment (5.2020) haven't found any better alternative to cmake, but there are multiple tools and ecosystems evolving in parallel to cmake, so need to observe what future might bring.
Then concerning packaging. choco seems to be a subset of nuget - it extends nuspec with it's own xml tags supporting software update. (referred web pages: https://chocolatey.org/, https://docs.microsoft.com/en-us/nuget/reference/nuspec)
If you want to publish nuget package or even installer package, it's possible to use nuget repository (without any extra costs).
If insufficient for your needs, it's also possible to upgrade nuget server to choco server, but it might costs something - see chocolatey web pages.
Api/abi breaks cannot be observed on any level, until actualy compilation / link error occurs or application crashes at run-time - only alternative it's possible to do - is to control versioning of packaging itself - so master package nuget(or choco) version would require higher version of dependent nuget (or choco) version.
If main repository is developed by same guys as child repository - then api/abi breaks can be done by developer itself, but if two gits are independent from each other and developed by different teams - then api/abi breaks can occur at any point of time.
Best way to ensure that repos are consistent - is to have unit testing on of master repository, which would check that no api/abi break occurs.
Theoretically if api/abi breaks occurs - build system itself could continue with two build alternatives -
- follow main repository with last working child repository
- follow child repository without building main repository
Same mechanics could be applied by using branches, e.g.
- "main git: master branch" + "child git: release/1.0 branch"
- "child git: master branch"
This is of mechanism might require to observe not only two repository's, but also switching branches in case of long lasting failures. (E.g. development team of child git does not care about build breaks occurring in main git)
Suspect there does not exists any ready made tools for this purpose (please leave comment if such will appear), as this might require reconnect tools from two potentially different build chains from potentially different git repositories, from potentially different organizations.
And last, but not least - if you want your application to be software update capable - then nuget/choco distribution download could be used with additional nuget/choco server.
Here is good example of download part for nuget package: https://github.com/mwrock/NugetDownloadFeed
Install package building is bit more complex issue - see for example following links:
(In order best-worse based on my opinion)
这篇关于在源代码和预编译二进制文件之间切换的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
