如何制作浅的 git 子模块? [英] How to make shallow git submodules?

问题描述

是否可以有浅层子模块?我有一个包含多个子模块的超级项目，每个子模块都有很长的历史，所以拖着所有的历史，它会变得不必要地大.

我所找到的只是这个未答复的线程.

我应该hack git-submodule 来实现这个?

解决方案

新的 git1.8.4(2013 年 7 月):

<块引用>

"git submodule update"可以选择浅克隆子模块存储库.

(并且 git 2.10 Q3 2016 允许使用 git config -f .gitmodules submodule..shallow true 记录.
请参阅此答案的结尾)

参见 commit 275cd184d52b5b81cb89e4ec332f60c:<块引用>

将 --depth 选项添加到git submodule"的添加和更新命令中，然后将其传递给克隆命令.当子模块很大并且除了最新提交之外你对任何事情都不感兴趣时​​，这很有用.

<块引用>

添加了测试并进行了一些缩进调整，以符合子模块更新可以处理 pwd 中的符号链接"上的测试文件的其余部分.

<块引用>

签字人:Fredrik Gustafsson <[e-mail]>
确认者:Jens Lehmann

这意味着这有效:

# 添加浅层子模块git submodule add --depth 1 <repo-url><路径>git config -f .gitmodules submodule..shallow true# 后来不浅git config -f .gitmodules submodule..shallow falsegit 子模块更新 <path>

命令可以按任何顺序运行.git submodule 命令执行实际的克隆(这次使用深度 1).并且 git config 命令使该选项对于稍后将递归克隆 repo 的其他人永久使用.

举个例子，假设你有 repo https://github.com/foo/bar 并且你想添加 https://github.com/lorem/ipsum 作为子模块，在您的存储库中 path/to/submodule.命令可能如下所示:

git submodule add --depth 1 [e-mail]:lorem/ipsum.git path/to/submodulegit config -f .gitmodules submodule.path/to/submodule.shallow true

以下结果也相同(顺序相反):

git config -f .gitmodules submodule.path/to/submodule.shallow truegit submodule add --depth 1 [e-mail]:lorem/ipsum.git path/to/submodule

下次有人运行 git clone --recursive [e-mail]:foo/bar.git 时，它会拉入 https://github.com 的整个历史/foo/bar，但它只会按预期浅克隆子模块.

与:

--深度

<块引用>

此选项对 add 和 update 命令有效.
创建一个浅"克隆，其历史记录被截断为指定数量的修订.

---

atwyman 添加了 在评论中:

<块引用>

据我所知，此选项不适用于不密切跟踪 master 的子模块.如果你设置深度 1，那么 submodule update 只有当你想要的子模块提交是最新的主模块时才能成功.否则你会得到fatal: reference is not a tree".

确实如此.
也就是说，直到 git 2.8(2016 年 3 月).在 2.8 中，submodule update --depth 还有一次成功的机会，即使 SHA1 可以从远程存储库 HEAD 之一直接访问.

参见 commit fb43e31(2016 年 2 月 24 日)Stefan Beller (stefanbeller).
帮助者:Junio C Hamano (gitster).
^{(由 Junio C Hamano 合并 -- gitster -- 在 提交 9671a76，2016 年 2 月 26 日)}

<块引用>

子模块:尝试通过直接获取 sha1 来尝试获取所需的 sha1

<块引用>

在审查同时更新 Gerrit 中的子模块的更改时，常见的审查做法是在本地下载并挑选补丁进行测试.
但是，在本地测试时，git submodule update"可能无法获取正确的子模块 sha1，因为子模块中的相应提交还不是项目历史的一部分，而只是提议的更改.

<块引用>

如果 $sha1 不是默认获取的一部分，我们会尝试直接获取 $sha1 .但是有些服务器不支持通过 sha1 直接获取，这会导致 git-fetch 快速失败.
我们可能会在这里失败，因为仍然缺少的 sha1 无论如何都会导致稍后在结账阶段失败，所以在这里失败是我们所能得到的.

---

MVG 指出 在评论中 到commit fb43e31 请求 SHA1 id 丢失的提交，所以服务器上的 uploadpack.allowReachableSHA1InWant 和 uploadpack.allowTipSHA1InWant 设置可能会影响这是否有效.
我今天写了一篇帖子到git列表，指出如何浅层子模块的使用可以在某些情况下更好地工作，即如果提交也是一个标签.
让我们拭目以待.

<块引用>

我想这就是 fb43e31 在获取默认分支后将特定 SHA1 的获取作为后备的原因.尽管如此，在--depth 1"的情况下，我认为提前中止是有意义的:如果列出的所有引用都与请求的引用都不匹配，并且服务器不支持 SHA1 请求，则没有意义获取任何东西，因为我们将无法满足子模块的要求.

---

2016 年 8 月更新(3 年后)

使用 Git 2.10(2016 年第三季度)，您将能够做到

 git config -f .gitmodules submodule..shallow true

参见没有额外重量的 Git 子模块"；了解更多.

---

Git 2.13(2017 年第二季度)添加 commit 8d3047c (19 A)作者:塞巴斯蒂安·舒伯特 (sschuberth).
^{(由 Sebastian Schuberth -- sschuberth -- 合并在 commit 8d3047c，2017 年 4 月 20 日)}

<块引用>

此子模块的克隆将作为浅克隆执行(历史深度为 1)

但是，Ciro Santilli 添加了在评论中(以及在他的回答中)

<块引用>

shallow = true on .gitmodules 仅影响使用 --recurse-submodules 时远程 HEAD 跟踪的引用，即使如果分支指向目标提交，即使您将 branch = mybranch 也放在 .gitmodules 上.

---

Git 2.20(2018 年第 4 季度)改进了子模块支持，当 .gitmodules 文件丢失时，子模块支持已更新为从 HEAD:.gitmodules 处的 blob 读取来自工作树.

参见 commit 2b1257e、commit 76e9bdc(2018 年 10 月 25 日)和 提交 b5c259f、提交 23dd8f5、"https://github.com/git/git/commit/b2faad44e2686cb996b9bf2a2dd22f787fc05f16" rel="noreferrer">commit b2faad4, 提交 2502ffc、提交 996df4dhref="https://github.com/git/git/commit/d1b13df5273bc6720c0f73f15efe87fa7c181f23" rel="noreferrer">提交 d1b13df，提交 45f5ef3、commit bcbc780(2018 年 10 月 5 日)由 Anao2).
^{(由 Junio C Hamano 合并 -- gitster -- 在 提交 abb4824，2018 年 11 月 13 日)}

<块引用>

submodule:当.gitmodules不在工作树中时支持读取

<块引用>

当 .gitmodules 文件在工作树中不可用时，尝试使用来自索引和当前分支的内容.
这涵盖了文件是存储库的一部分但对于某些未结帐的原因，例如结帐稀疏.

<块引用>

这使得至少可以使用git submodule"命令读取 gitmodules 配置文件而不完全填充工作树.

<块引用>

写入 .gitmodules 仍然需要检出文件，所以在调用 config_set_in_gitmodules_file_gently 之前检查一下.

<块引用>

也在 git-submodule.sh::cmd_add() 中添加类似的检查以预测git submodule add"的最终失败；.gitmodules 不能安全写入时的命令；这可以防止命令使存储库处于虚假状态(例如，子模块存储库已克隆但 .gitmodules 未更新，因为 config_set_in_gitmodules_file_gently 失败).

<块引用>

此外，由于 config_from_gitmodules() 现在访问全局对象存储，需要保护所有调用该函数的代码路径防止并发访问全局对象存储.
目前这只发生在 builtin/grep.c::grep_submodules() 中，所以调用grep_read_lock() 在调用涉及 config_from_gitmodules() 的代码之前.

<块引用>

注意:此新功能在极少数情况下不起作用还没有错:工作树中没有 .gitmodules 的嵌套子模块.

---

注意:Git 2.24(2019 年第四季度)修复了克隆浅层子模块时可能出现的段错误.

参见 commit ddb3c85(2019 年 9 月 30 日)Ali Utku Selen (auselen).
^{(由 Junio C Hamano 合并 -- gitster -- 在 commit 678a9ca，2019 年 10 月 9 日)}

---

Git 2.25(2020 年第一季度)，阐明了 git submodule update 文档.

参见 commit f0e58b3(2019 年 11 月 24 日)菲利普布莱恩 (phil-blain).
^{(由 Junio C Hamano 合并 -- gitster -- 在 提交 ef61045，2019 年 12 月 5 日)}

<块引用>

doc:提到'git submodule update'获取丢失的提交

^{帮助:Junio C Hamano
帮助:约翰内斯·辛德林
签字人:Philippe Blain}

<块引用>

'git submodule 更新'将如果未找到记录在超级项目中的 SHA-1，则从远程子模块获取新提交.文档中没有提到这一点.

---

警告:在 Git 2.25(2020 年第 1 季度)中，git clone --recurse-submodules"之间的交互将不再有效.和备用对象存储设计不当.

<块引用>

文档和代码被教导在用户看到失败时提供更清晰的建议.

参见 commit 4f3e57e、commit 10c64a0(2019 年 12 月 2 日)由 jhowtan).
^{(由 Junio C Hamano 合并 -- gitster -- 在 提交 5dd1d59，2019 年 12 月 10 日)}

<块引用>

submodule--helper:关于致命替代的建议错误

^{签字人:Jonathan Tan
确认:Jeff King}

<块引用>

当递归克隆一个超级项目时，在它的 .gitmodules 中定义了一些浅层模块，然后用--reference="重新克隆时，一个错误发生.例如:

<块引用>

git clone --recurse-submodules --branch=master -j8 

 https://android.googlesource.com/platform/superproject 掌握git clone --recurse-submodules --branch=master -j8 https://android.googlesource.com/platform/superproject --参考大师master2

<块引用>

失败:

<块引用>

fatal: 子模块 ''无法添加替代:参考存储库

''很浅

<块引用>

当无法添加从超级项目的替代项计算的替代项时，无论是在这种情况下还是其他情况，建议配置submodule.alternateErrorStrategy"；配置选项并使用--reference-if-able"；而不是--reference"；克隆时.

详情见:

使用 Git 2.25(2020 年第一季度)，git clone --recurse-submodules"之间的交互和备用对象存储设计不当.

<块引用>

Doc:解释 submodule.alternateErrorStrategy

^{签字人:Jonathan Tan
确认:Jeff King}

<块引用>

提交 31224cbdc7(clone:递归和参考选项触发子模块替代", 2016-08-17, Git v2.11.0-rc0 -- 列在 batch #1) 教导 Git 支持配置选项 "submodule.alternateLocation"和submodule.alternateErrorStrategy"；在一个超级项目上.

<块引用>

如果submodule.alternateLocation"被配置为superproject"；在超级项目上，每当该超级项目的子模块被克隆时，它都会从超级项目的 $GIT_DIR/objects/info/alternates 计算该子模块的类似替代路径，并引用它.

<块引用>

submodule.alternateErrorStrategy"选项确定如果无法引用该替代项会发生什么.
然而，当该选项未设置为死亡"时，克隆是否像未指定替代项一样继续进行并不清楚.(在 31224cbdc7 中的测试中可以看出).
因此，请相应地记录下来.

配置子模块文档现在包括:

<块引用>

submodule.alternateErrorStrategy::

<块引用>

指定如何使用通过 submodule.alternateLocation 计算的子模块的替代项来处理错误.
可能的值为 ignore、info、die.
默认为 die.
请注意，如果设置为 ignore 或 info，并且如果计算出的替代项有错误，则克隆会像未指定替代项一样继续.

---

注意:"git 子模块更新--quiet"^(man) 没有将安静选项传播到底层 git fetch^(man)，已在 Git 2.32(2021 年第二季度)中得到更正.

参见 commit 62af4bd(2021 年 4 月 30 日)尼古拉斯·克拉克 (nwc10).
^{(由 Junio C Hamano 合并 -- gitster -- 在 提交 74339f8，2021 年 5 月 11 日)}

<块引用>

子模块更新:使用 "--安静"

^{签字人:尼古拉斯·克拉克}

<块引用>

命令如

$ git submodule update --quiet --init --depth=1

涉及浅层克隆，调用shell函数fetch_in_submodule，又调用git fetch.
在那里传递 --quiet 选项.

Is it possible to have shallow submodules? I have a superproject with several submodules, each with a long history, so it gets unnecessarily big dragging all that history.

All I have found is this unanswered thread.

Should I just hack git-submodule to implement this?

解决方案

New in the upcoming git1.8.4 (July 2013):

"git submodule update" can optionally clone the submodule repositories shallowly.

(And git 2.10 Q3 2016 allows to record that with git config -f .gitmodules submodule.<name>.shallow true.
See the end of this answer)

See commit 275cd184d52b5b81cb89e4ec33e540fb2ae61c1f:

Add the --depth option to the add and update commands of "git submodule", which is then passed on to the clone command. This is useful when the submodule(s) are huge and you're not really interested in anything but the latest commit.

Tests are added and some indention adjustments were made to conform to the rest of the testfile on "submodule update can handle symbolic links in pwd".

Signed-off-by: Fredrik Gustafsson <[e-mail]>
Acked-by: Jens Lehmann <[e-mail]>

That means this works:

# add shallow submodule
git submodule add --depth 1 <repo-url> <path>
git config -f .gitmodules submodule.<path>.shallow true

# later unshallow
git config -f .gitmodules submodule.<path>.shallow false
git submodule update <path>

The commands can be ran in any order. The git submodule command perform the actual clone (using depth 1 this time). And the git config commands make the option permanent for other people who will clone the repo recursively later.

As an example, suppose you have the repo https://github.com/foo/bar and you want to add https://github.com/lorem/ipsum as a submodule, in your repo at path/to/submodule. The commands may look like like the following:

git submodule add --depth 1 [e-mail]:lorem/ipsum.git path/to/submodule
git config -f .gitmodules submodule.path/to/submodule.shallow true

The following results in the same thing too (opposite order):

git config -f .gitmodules submodule.path/to/submodule.shallow true
git submodule add --depth 1 [e-mail]:lorem/ipsum.git path/to/submodule

The next time someone runs git clone --recursive [e-mail]:foo/bar.git, it will pull in the whole history of https://github.com/foo/bar, but it will only shallow-clone the submodule as expected.

With:

--depth

This option is valid for add and update commands.
Create a 'shallow' clone with a history truncated to the specified number of revisions.

---

atwyman adds in the comments:

As far as I can tell this option isn't usable for submodules which don't track master very closely. If you set depth 1, then submodule update can only ever succeed if the submodule commit you want is the latest master. Otherwise you get "fatal: reference is not a tree".

That is true.
That is, until git 2.8 (March 2016). With 2.8, the submodule update --depth has one more chance to succeed, even if the SHA1 is directly reachable from one of the remote repo HEADs.

See commit fb43e31 (24 Feb 2016) by Stefan Beller (stefanbeller).
Helped-by: Junio C Hamano (gitster).
^{(Merged by Junio C Hamano -- gitster -- in commit 9671a76, 26 Feb 2016)}

submodule: try harder to fetch needed sha1 by direct fetching sha1

When reviewing a change that also updates a submodule in Gerrit, a common review practice is to download and cherry-pick the patch locally to test it.
However when testing it locally, the 'git submodule update' may fail fetching the correct submodule sha1 as the corresponding commit in the submodule is not yet part of the project history, but also just a proposed change.

If $sha1 was not part of the default fetch, we try to fetch the $sha1 directly. Some servers however do not support direct fetch by sha1, which leads git-fetch to fail quickly.
We can fail ourselves here as the still missing sha1 would lead to a failure later in the checkout stage anyway, so failing here is as good as we can get.

---

MVG points out in the comments to commit fb43e31 (git 2.9, Feb 2016)

It would seem to me that commit fb43e31 requests the missing commit by SHA1 id, so the uploadpack.allowReachableSHA1InWant and uploadpack.allowTipSHA1InWant settings on the server will probably affect whether this works.
I wrote a post to the git list today, pointing out how the use of shallow submodules could be made to work better for some scenarios, namely if the commit is also a tag.
Let's wait and see.

I guess this is a reason why fb43e31 made the fetch for a specific SHA1 a fallback after the fetch for the default branch.
Nevertheless, in case of "--depth 1" I think it would make sense to abort early: if none of the listed refs matches the requested one, and asking by SHA1 isn't supported by the server, then there is no point in fetching anything, since we won't be able to satisfy the submodule requirement either way.

---

Update August 2016 (3 years later)

With Git 2.10 (Q3 2016), you will be able to do

 git config -f .gitmodules submodule.<name>.shallow true

See "Git submodule without extra weight" for more.

---

Git 2.13 (Q2 2017) do add in commit 8d3047c (19 Apr 2017) by Sebastian Schuberth (sschuberth).
^{(Merged by Sebastian Schuberth -- sschuberth -- in commit 8d3047c, 20 Apr 2017)}

a clone of this submodule will be performed as a shallow clone (with a history depth of 1)

However, Ciro Santilli adds in the comments (and details in his answer)

shallow = true on .gitmodules only affects the reference tracked by the HEAD of the remote when using --recurse-submodules, even if the target commit is pointed to by a branch, and even if you put branch = mybranch on the .gitmodules as well.

---

Git 2.20 (Q4 2018) improves on the submodule support, which has been updated to read from the blob at HEAD:.gitmodules when the .gitmodules file is missing from the working tree.

See commit 2b1257e, commit 76e9bdc (25 Oct 2018), and commit b5c259f, commit 23dd8f5, commit b2faad4, commit 2502ffc, commit 996df4d, commit d1b13df, commit 45f5ef3, commit bcbc780 (05 Oct 2018) by Antonio Ospite (ao2).
^{(Merged by Junio C Hamano -- gitster -- in commit abb4824, 13 Nov 2018)}

submodule: support reading .gitmodules when it's not in the working tree

When the .gitmodules file is not available in the working tree, try using the content from the index and from the current branch.
This covers the case when the file is part of the repository but for some reason it is not checked out, for example because of a sparse checkout.

This makes it possible to use at least the 'git submodule' commands which read the gitmodules configuration file without fully populating the working tree.

Writing to .gitmodules will still require that the file is checked out, so check for that before calling config_set_in_gitmodules_file_gently.

Add a similar check also in git-submodule.sh::cmd_add() to anticipate the eventual failure of the "git submodule add" command when .gitmodules is not safely writeable; this prevents the command from leaving the repository in a spurious state (e.g. the submodule repository was cloned but .gitmodules was not updated because config_set_in_gitmodules_file_gently failed).

Moreover, since config_from_gitmodules() now accesses the global object store, it is necessary to protect all code paths which call the function against concurrent access to the global object store.
Currently this only happens in builtin/grep.c::grep_submodules(), so call grep_read_lock() before invoking code involving config_from_gitmodules().

NOTE: there is one rare case where this new feature does not work properly yet: nested submodules without .gitmodules in their working tree.

---

Note: Git 2.24 (Q4 2019) fixes a possible segfault when cloning a submodule shallow.

See commit ddb3c85 (30 Sep 2019) by Ali Utku Selen (auselen).
^{(Merged by Junio C Hamano -- gitster -- in commit 678a9ca, 09 Oct 2019)}

---

Git 2.25 (Q1 2020), clarifies the git submodule update documentation.

See commit f0e58b3 (24 Nov 2019) by Philippe Blain (phil-blain).
^{(Merged by Junio C Hamano -- gitster -- in commit ef61045, 05 Dec 2019)}

doc: mention that 'git submodule update' fetches missing commits

^{Helped-by: Junio C Hamano
Helped-by: Johannes Schindelin
Signed-off-by: Philippe Blain}

'git submodule update' will fetch new commits from the submodule remote if the SHA-1 recorded in the superproject is not found. This was not mentioned in the documentation.

---

Warning: With Git 2.25 (Q1 2020), the interaction between "git clone --recurse-submodules" and alternate object store was ill-designed.

The documentation and code have been taught to make more clear recommendations when the users see failures.

See commit 4f3e57e, commit 10c64a0 (02 Dec 2019) by Jonathan Tan (jhowtan).
^{(Merged by Junio C Hamano -- gitster -- in commit 5dd1d59, 10 Dec 2019)}

submodule--helper: advise on fatal alternate error

^{Signed-off-by: Jonathan Tan
Acked-by: Jeff King}

When recursively cloning a superproject with some shallow modules defined in its .gitmodules, then recloning with "--reference=<path>", an error occurs. For example:

git clone --recurse-submodules --branch=master -j8 

  https://android.googlesource.com/platform/superproject 
  master
git clone --recurse-submodules --branch=master -j8 
  https://android.googlesource.com/platform/superproject 
  --reference master master2

fails with:

fatal: submodule '<snip>' cannot add alternate: reference repository

'<snip>' is shallow

When a alternate computed from the superproject's alternate cannot be added, whether in this case or another, advise about configuring the "submodule.alternateErrorStrategy" configuration option and using "--reference-if-able" instead of "--reference" when cloning.

That is detailed in:

With Git 2.25 (Q1 2020), The interaction between "git clone --recurse-submodules" and alternate object store was ill-designed.

Doc: explain submodule.alternateErrorStrategy

^{Signed-off-by: Jonathan Tan
Acked-by: Jeff King}

Commit 31224cbdc7 ("clone: recursive and reference option triggers submodule alternates", 2016-08-17, Git v2.11.0-rc0 -- merge listed in batch #1) taught Git to support the configuration options "submodule.alternateLocation" and "submodule.alternateErrorStrategy" on a superproject.

If "submodule.alternateLocation" is configured to "superproject" on a superproject, whenever a submodule of that superproject is cloned, it instead computes the analogous alternate path for that submodule from $GIT_DIR/objects/info/alternates of the superproject, and references it.

The "submodule.alternateErrorStrategy" option determines what happens if that alternate cannot be referenced.
However, it is not clear that the clone proceeds as if no alternate was specified when that option is not set to "die" (as can be seen in the tests in 31224cbdc7).
Therefore, document it accordingly.

The config submodule documentation now includes:

submodule.alternateErrorStrategy::

Specifies how to treat errors with the alternates for a submodule as computed via submodule.alternateLocation.
Possible values are ignore, info, die.
Default is die.
Note that if set to ignore or info, and if there is an error with the computed alternate, the clone proceeds as if no alternate was specified.

---

Note: "git submodule update --quiet"^(man) did not propagate the quiet option down to underlying git fetch^(man), which has been corrected with Git 2.32 (Q2 2021).

See commit 62af4bd (30 Apr 2021) by Nicholas Clark (nwc10).
^{(Merged by Junio C Hamano -- gitster -- in commit 74339f8, 11 May 2021)}

submodule update: silence underlying fetch with "--quiet"

^{Signed-off-by: Nicholas Clark}

Commands such as

$ git submodule update --quiet --init --depth=1

involving shallow clones, call the shell function fetch_in_submodule, which in turn invokes git fetch.
Pass the --quiet option onward there.

这篇关于如何制作浅的 git 子模块?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！