将大型仓库划分为多个子仓库并保存历史记录(Mercurial) [英] Split large repo into multiple subrepos and preserve history (Mercurial)

问题描述

我们有大量的代码库，其中在SVN的一个目录中包含多个共享项目，解决方案文件等.我们正在迁移到Mercurial.我想借此机会将我们的代码重组为几个存储库，以使克隆分支的开销更少.在保存历史记录的同时，我已经成功地将我们的仓库从SVN转换为Mercurial.我的问题是:如何在保留其历史记录的同时将所有不同的项目分解为单独的存储库?

以下是我们的单个存储库(OurPlatform)当前的示例:

/OurPlatform
---- Core
---- Core.Tests
---- Database
---- Database.Tests
---- CMS
---- CMS.Tests
---- Product1.Domain
---- Product1.Stresstester
---- Product1.Web
---- Product1.Web.Tests
---- Product2.Domain
---- Product2.Stresstester
---- Product2.Web
---- Product2.Web.Tests
==== Product1.sln
==== Product2.sln

除解决方案文件外，所有这些都是包含VS Projects的文件夹. Product1.sln和Product2.sln都引用所有其他项目.理想情况下，我想将每个文件夹都带走，然后将它们转换为单独的Hg仓库，并为每个项目添加新的仓库(它们将作为父仓库).然后，如果有人要处理Product1，他们将克隆Product1存储库，其中包含Product1.sln和对ReferenceAssemblies，Core，Core.Tests，Database，Database.Tests，CMS和CMS.Tests的子存储库引用. >

因此，只需在项目目录中进行hg init即可轻松做到这一点.但是在保存历史的同时还能做到吗?还是有更好的方法来安排这个?

EDIT ::::

由于Ry4an的回答，我得以实现自己的目标.我想分享我在这里为别人做的事.

由于我们有很多单独的项目，因此我编写了一个小的bash脚本来自动创建文件映射并创建最终的bat脚本来实际执行转换.从答案中不能完全看出，对于每个文件映射，必须为每个文件映射运行一次convert命令，以为每个项目生成一个单独的存储库.该脚本将放置在您先前已转换的svn工作副本上方的目录中.我使用了工作副本，因为它的文件结构与我希望最终的新hg存储库最匹配.

#!/bin/bash

# this requires you to be in: /path/to/svn/working/copy/, and issue: ../filemaplister.sh ./

for filename in *
do
  extension=${filename##*.} #$filename|awk -F . '{print $NF}'
  if [ "$extension" == "sln" -o "$extension" == "suo" -o "$extension" == "vsmdi" ]; then
    base=${filename%.*}
    echo "#$base.filemap" >> "$base.filemap"
    echo "include $filename" >> "$base.filemap"
    echo "C:\Applications\TortoiseHgPortable\hg.exe convert --filemap $base.filemap ../hg-datesort-converted ../hg-separated/$base > $base.convert.output.txt" >> "MASTERGO.convert.bat"
  else
    echo "#$filename.filemap" >> "$filename.filemap"
    echo "include $filename" >> "$filename.filemap"
    echo "rename $filename ." >> "$filename.filemap"
    echo "C:\Applications\TortoiseHgPortable\hg.exe convert --filemap $filename.filemap ../hg-datesort-converted ../hg-separated/$filename > $filename.convert.output.txt" >> "MASTERGO.convert.bat"  
  fi  
done;

mv *.filemap ../hg-conversion-filemaps/
mv *.convert.bat ../hg-conversion-filemaps/

此脚本查看svn工作副本中的每个文件，并根据类型创建新的文件映射文件或追加到现有文件映射文件. if实际上只是用来捕获其他Visual Studio文件，并将它们放置在单独的存储库中.这本应在bash上运行(在我的情况下为cygwin)，但是由于Windows上的分叉/进程问题，运行实际的convert命令是通过TortoiseHg随附的hg版本完成的(gah，我知道...).

因此，您运行MASTERGO.convert.bat文件，该文件将查看转换后的hg存储库，并使用提供的文件映射创建单独的存储库.完成后，将有一个名为hg-separated的文件夹，其中包含每个项目的文件夹/存储库以及每个解决方案的文件夹/存储库.然后，您必须手动将所有项目克隆到解决方案存储库中，并将克隆添加到.hgsub文件中.提交后，将创建一个.hgsubstate文件，您就可以开始了！

使用上面的示例，我的.hgsub文件对于"Product1"如下所示:

Product1.Domain = /absolute/path/to/Product1.Domain
Product1.Stresstester = /absolute/path/to/Product1.Stresstester
Product1.Web = /absolute/path/to/Product1.Web
Product1.Web.Tests = /absolute/path/to/Product1.Web.Tests

将这些存储库转移到中央服务器后，我将手动将路径更改为url.

而且，由于现在所有内容都已分离，因此没有类似于最初的OurPlatform svn存储库的类似物.

再次感谢！

解决方案

这绝对可以做到.您将要使用hg convert命令.这是我要使用的过程:

1. 使用hg convert将所有内容转换为单个hg存储库，源类型为svn，目标类型为hg(听起来您已经完成了此步骤)
2. 创建filemap个文件的集合，以与hg convert的--filemap选项一起使用
使用源类型为hg和目标类型为hg的
3. 运行hg convert，并且源是在第一步中创建的Mercurial存储库-并对在第二步中创建的每个文件映射执行此操作.

文件映射语法显示在hg help convert输出中，但这是要点:

The filemap is a file that allows filtering and remapping of files and
directories. Comment lines start with '#'. Each line can contain one of
the following directives:

  include path/to/file

  exclude path/to/file

  rename from/file to/file

因此，在您的示例中，文件映射如下所示:

# this is Core.filemap
include Core
rename Core .

请注意，如果包含，则表示排除了所有其他内容.同样，重命名行以点结尾，并将所有内容上移一层.

# this is Core.Tests
include Core.Tests
rename Core.Tests .

以此类推.

为每个新存储库创建中断存储库后，可以删除在第一步中创建的所有初始存储库，然后开始在.hgsub文件中设置子存储库配置.

We have a large base of code that contains several shared projects, solution files, etc in one directory in SVN. We're migrating to Mercurial. I would like to take this opportunity to reorganize our code into several repositories to make cloning for branching have less overhead. I've already successfully converted our repo from SVN to Mercurial while preserving history. My question: how do I break all the different projects into separate repositories while preserving their history?

Here is an example of what our single repository (OurPlatform) currently looks like:

/OurPlatform
---- Core
---- Core.Tests
---- Database
---- Database.Tests
---- CMS
---- CMS.Tests
---- Product1.Domain
---- Product1.Stresstester
---- Product1.Web
---- Product1.Web.Tests
---- Product2.Domain
---- Product2.Stresstester
---- Product2.Web
---- Product2.Web.Tests
==== Product1.sln
==== Product2.sln

All of those are folders containing VS Projects except for the solution files. Product1.sln and Product2.sln both reference all of the other projects. Ideally, I'd like to take each of those folders, and turn them into separate Hg repos, and also add new repos for each project (they would act as parent repos). Then, If someone was going to work on Product1, they would clone the Product1 repo, which contained Product1.sln and subrepo references to ReferenceAssemblies, Core, Core.Tests, Database, Database.Tests, CMS, and CMS.Tests.

So, it's easy to do this by just hg init'ing in the project directories. But can it be done while preserving history? Or is there a better way to arrange this?

EDIT::::

Thanks to Ry4an's answer, I was able to accomplish my goal. I wanted to share how I did it here for others.

Since we had a lot of separate projects, I wrote a small bash script to automate creating the filemaps and to create the final bat script to actually do the conversion. What wasn't completely apparent from the answer, is that the convert command needs to be run once for each filemap, to produce a separate repository for each project. This script would be placed in the directory above a svn working copy that you have previously converted. I used the working copy since it's file structure best matched what I wanted the final new hg repos to be.

#!/bin/bash

# this requires you to be in: /path/to/svn/working/copy/, and issue: ../filemaplister.sh ./

for filename in *
do
  extension=${filename##*.} #$filename|awk -F . '{print $NF}'
  if [ "$extension" == "sln" -o "$extension" == "suo" -o "$extension" == "vsmdi" ]; then
    base=${filename%.*}
    echo "#$base.filemap" >> "$base.filemap"
    echo "include $filename" >> "$base.filemap"
    echo "C:\Applications\TortoiseHgPortable\hg.exe convert --filemap $base.filemap ../hg-datesort-converted ../hg-separated/$base > $base.convert.output.txt" >> "MASTERGO.convert.bat"
  else
    echo "#$filename.filemap" >> "$filename.filemap"
    echo "include $filename" >> "$filename.filemap"
    echo "rename $filename ." >> "$filename.filemap"
    echo "C:\Applications\TortoiseHgPortable\hg.exe convert --filemap $filename.filemap ../hg-datesort-converted ../hg-separated/$filename > $filename.convert.output.txt" >> "MASTERGO.convert.bat"  
  fi  
done;

mv *.filemap ../hg-conversion-filemaps/
mv *.convert.bat ../hg-conversion-filemaps/

This script looks at every file in an svn working copy, and depending on the type either creates a new filemap file or appends to an existing one. The if is really just to catch misc visual studio files, and place them into a separate repo. This is meant to be run on bash (cygwin in my case), but running the actual convert command is accomplished through the version of hg shipped with TortoiseHg due to forking/process issues on Windows (gah, I know...).

So you run the MASTERGO.convert.bat file, which looks at your converted hg repo, and creates separate repos using the supplied filemap. After it is complete, there is a folder called hg-separated that contains a folder/repo for each project, as well as a folder/repo for each solution. You then have to manually clone all the projects into a solution repo, and add the clones to the .hgsub file. After committing, an .hgsubstate file is created and you're set to go!

With the example given above, my .hgsub file looks like this for "Product1":

Product1.Domain = /absolute/path/to/Product1.Domain
Product1.Stresstester = /absolute/path/to/Product1.Stresstester
Product1.Web = /absolute/path/to/Product1.Web
Product1.Web.Tests = /absolute/path/to/Product1.Web.Tests

Once I transfer these repos to a central server, I'll be manually changing the paths to be urls.

Also, there is no analog to the initial OurPlatform svn repo, since everything is separated now.

Thanks again!

解决方案

This can absolutely be done. You'll want to use the hg convert command. Here's the process I'd use:

1. convert everything to a single hg repository using hg convert with a source type of svn and a dest type of hg (it sounds like you've already done this step)
2. create a collection of filemap files for use with hg convert's --filemap option
3. run hg convert with source type hg and dest type hg and the source being the mercurial repo created in step one -- and do it for each of the filemaps you created in step two.

The filemap syntax is shown in the hg help convert output, but here's the gist:

The filemap is a file that allows filtering and remapping of files and
directories. Comment lines start with '#'. Each line can contain one of
the following directives:

  include path/to/file

  exclude path/to/file

  rename from/file to/file

So in your example your filemaps would look like this:

# this is Core.filemap
include Core
rename Core .

Note that if you have an include that the exclusion of everything else is implied. Also that rename line ends in a dot and moves everything up one level.

# this is Core.Tests
include Core.Tests
rename Core.Tests .

and so on.

Once you've created the broken-out repositories for each of the new repos, you can delete the has-everything initial repo created in step one and start setting up your subrepo configuration in .hgsub files.

这篇关于将大型仓库划分为多个子仓库并保存历史记录(Mercurial)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！