管理源代码管理下的代码使用的第三方来源和二进制文件 [英] Managing 3rd party sources and binaries used by code under source control

问题描述

我在源代码控制下有一个很大的代码库（是subversion，现在是git）。为了编译代码并运行测试，我使用了一组第三方库。这些库可分为几类

• 仅限二进制文件


• 第三方来源


• 第三方来源和本地修改

每个库都有它的{Windows，Linux} X {debug，release } X {32位，64位}配置。此外，这些库随着时间的推移而不断发展，我的项目的不同版本使用这些库的不同版本/版本。

我的问题是存储的最佳方式是什么这些第三方？

以下是我的偏好设置：

1. 保持项目源代码库的大小很小


2. 保持项目源与第三方同步，这样我就可以随时编译和运行旧版本


3. 简单管理


4. 跨平台

我尝试了几种解决方案，也不令人满意：

1. 使用版本化脚本从手持管理的ftp服务器获取二进制文件，该服务器包含所有版本的库。这可行，但需要仔细管理服务器上的目录结构。这很容易出错，因为有人可能用新版本覆盖其中一个二进制文件。


2. SVN外部 - 当时SVN外部程序无法引用特定标记。今天我使用的是git。


3. Git子模块 - 提取可能很大的整个外部存储库。或者，它需要为每个单独的库管理一个单独的存储库。子模块指向一个特定的标签，这意味着当我只需要一些外部设备时，或者我在git树中模仿一些奇怪的文件系统时，就可以获得所有外部设备。

很明显，第三方资源需要存储在供应商分支的git中，但二进制文件和头文件则是另一回事。

解决方案

我的问题的公平解决方案是 git-subtree ，这是最近合并成主线git。它在我的要求和平台限制之间提供了一个合理的平衡。现在我有多个外部存储库（每个存储库都有一个供应商分支以及本地更改分支），每个项目存储库都将这些外部存储的一部分分为子文件夹。为了保持组织的有序性，我维护一个'bin'和'lib'文件夹，其中包含指向外部子文件夹中相应文件/文件夹的软链接。

git-subtree允许将外部资源库中的子树合并到子文件夹中。子文件夹可以与外部存储库来回合并。

优点/缺点：

1. 小型存储库 - 存储库并不像我希望的那么小，但它只包含外部存储库中必要的部分。为了节省空间，我尽量减少外部树木。我认为这是一个很好的代价，作为回报，我会获得简单和健壮的体验;因为加载和更新项目是一个简单的git pull，所有与项目相关的数据都包含在一个存储库中。项目/外部同步 - 作为项目和外部项目在同一个版本库中进行版本控制，我可以检出我想要的任何分支/标签，并期望它能够正常工作。

简单性 - 每天的工作是直截了当。更新外部存储库，创建一个新的存储库或切换到不同版本的外部可能会很棘手，需要特殊的语法。但是这确实发生得太多了。最好的事情是，你可以首先添加一个新的外部项目，然后再将它分割（使用git-subtree）到它自己的仓库中。 平台 - 它是git

2. 二进制文件 - 我决定避免保留二进制文件并提供Makefile。我做出这个决定是因为我的一些外部依赖于其他外部因素，这使得构建一个不经常改变的二进制文件非常困难。对于一些外部存储来说，由于构建时间很长，我确实存储了二进制文件。

结构：

  / root 
 / External 
 / External1（来自[e-mail]的git-subtree：External1 v1.0）
 / External2（来自[e-mail]的git-subtree：External2 v0.7）
 / lib 
 /libExternal1.a  - > ../External/External1/libExternal1.a 
 /libExternal2.a  - > ../External/External1/libExternal2.a 
 / include 
 / External1  - > ../External/External1/include 
 / External2  - > ../External/External2/include 
  

I have a large code base under source control (was subversion, now git). To compile the code and run the tests I use a set of 3rd party libraries. These libraries can be divided into few categoriesL

• Binaries only
• 3rd party sources
• 3rd party sources + local modifications

Each library has its {Windows, Linux} X {debug, release} X {32bit, 64bit} configurations. In addition these libraries evolve with time and different versions of my project use different versions/builds of these libraries.

My question is what is the best way to store these 3rd parties?

Here is my set of preferences:

1. Keep the size of the project source repository small
2. Keep the project source in sync with the 3rd parties so I can always compile and run and old version
3. Simple to manage
4. Cross platform

I tried and thought of several solutions but neither was satisfactory:

1. Use a versioned script to fetch the binaries from a manually managed ftp server that holds all versions of the libraries. This works, but requires careful management of the directory structure on the server. It is error prone as someone might overwrite one of the binaries with a new build.
2. SVN externals - At the time SVN externals could not refer to a specific tag. Today I am using git.
3. Git submodules - Pulls the entire external repository which can be huge. Alternatively it requires managing a separate repository for every single library. The submodule points at a specific tag which means either I get all the externals, when I need only some, or I mimic some weird file system in the git tree.

It is clear to me that the 3rd party sources need to be stored in git in a vendor branch, but the binaries and headers are a different story.

解决方案

A fair solution for my problem is git-subtree which was recently merged into mainline git. It provides a fair balance between my requirements and the platform limitations. Now I have multiple repositories for the externals (each has a vendor branch as well as local changes branch) and each project repository takes parts of these externals into sub-folders. To keep things organized I maintain a 'bin' and 'lib' folders which contains soft links to the appropriate files/folders in the externals sub-folder.

git-subtree allows to merge a sub-tree from an external repository into a sub-folder. The sub-folder can be merged back and forth with the external repository.

Pros/Cons:

1. Small repository - The repository is not as small as I would like it to be but it contains only the necessary parts from the external repositories. To save space I try to keep the external trees small. I consider it a good price to pay when in return I get simplicity and robustness; as loading and updating a project is a simple git pull and all project related data is contained in a single repository

2. Project/Externals sync - As the project and externals are versioned in the same repository, I can checkout any branch/tag I want and expect it to be working.

3. Simplicity - Day-by-day work is straight forward. Updating external repository, creating a new one or switching to a different version of the external may be tricky and requires special syntax. However this does happen too much. The best thing is that one can add a new external to this project first and only afterwards split it (using git-subtree) into its own repository.

4. Cross platform - Well it's git

5. Binaries - I decided to avoid holding binaries and provide Makefiles instead. I came to this decision as some of my externals depend on other externals which makes it very hard to build a binary that doesn't change very often. For some externals I do store the binaries due to very long build times.

Structure:

/root
   /External
      /External1 (git-subtree from [e-mail]:External1 v1.0)
      /External2 (git-subtree from [e-mail]:External2 v0.7)
   /lib
      /libExternal1.a -> ../External/External1/libExternal1.a
      /libExternal2.a -> ../External/External1/libExternal2.a
   /include
      /External1 -> ../External/External1/include
      /External2 -> ../External/External2/include

这篇关于管理源代码管理下的代码使用的第三方来源和二进制文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！