Git gc占用大量内存，即使我限制了它 [英] Git gc uses a lot of memory, even I limited it
本文介绍了Git gc占用大量内存，即使我限制了它的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我在共享主机(uberspace.de)上运行git，并且在执行git push或git gc时达到内存限制.他们报告的内存消耗约为1500 MB.
I am running git on a shared host (uberspace.de) and hit the memory limit when doing git push or git gc. Memory consumption reported by them is around 1500 MB.
所以我遵循了
https://stackoverflow.com/a/8761464/288568
它似乎可以工作一次，但是经过多次提交后，它不再起作用了，内存消耗再次达到1500MB左右，并且git pack进程被杀死了.
It seemed to work once, but after lots of commits it does not help anymore and the memory consumption is again around 1500MB and the git pack process gets killed.
我提交了一个120 MB的SQL转储，几乎没有什么更改(每次大约5行)-因此打包的存储库只有150 MB左右，但解压缩后只有3 GB(并且我达到了bitbucket的限制).
I commit a 120 MB SQL dump with very fews changes (around 5 lines each time) - so the packed repository is around 150 MB only, but unpacked around 3 GB (and I hit bitbucket's limit).
我看到哪个pmap打开了很多pack对象-累加起来是不是很大?如何限制呢?
Which pmap I see it is opening a lot of pack objects - is it that which adds up to the high memory? How can this be limited?
最后，我收到打包对象因信号15而死亡"和来自托管者的邮件，表明我的进程已被杀死.
Finally I get "pack-objects died of signal 15" and a mail from the hoster, that my process was killed.
sh-4.1$ git config --list|grep pack
pack.windowmemory=25m
pack.packsizelimit=25m
pack.threads=1
pack.deltecachesize=25m
sh-4.1$ git gc
Counting objects: 2586, done.
^Zmpressing objects: 15% (163/1085)
[1]+ Stopped(SIGTSTP) git gc
sh-4.1$ bg
[1] git gc &
sh-4.1$ pmap 14190
14190: git pack-objects --keep-true-parents --honor-pack-keep --non-empty --all --reflog --indexed-objects --unpack-unreachable=2.weeks.ago --local --delta-base-offset .git/objects/pack/.tmp-14187-pack
0000000000400000 1848K r-x-- /home/foo/.toast/pkg/git/v2.11.1/1/root/libexec/git-core/git
00000000007ce000 36K rw--- /home/foo/.toast/pkg/git/v2.11.1/1/root/libexec/git-core/git
00000000007d7000 280K rw--- [ anon ]
0000000000da1000 1680K rw--- [ anon ]
00007f735dc5d000 161648K rw--- [ anon ]
00007f7377555000 149580K rw--- [ anon ]
00007f73915ee000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-be75b796793b738f7566a4be06491cfd3ad9c236.pack
00007f73990b6000 125592K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-33af25946a2fcb1c3e284b3093ac0eafa440cf46.pack
00007f73a0b5c000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-dc37fbdaf5ad7f6c317a7c3d5ffecf53daa045b6.pack
00007f73a8624000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-c7d6e0b1e4da6a2aaa5eff3cad27f6fcbb38a5c9.pack
00007f73b00ec000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-bbef922979f633183f0bc207b5e52608b36d6434.pack
00007f73b7bb4000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-1468b8f7d29c999297b2411f9b94d86b0cd9965d.pack
00007f73bf67c000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-3ecc274dd42f4783926224457225968b1fe1d95b.pack
00007f73c7144000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-bb77ce8e83062bf2d26a3366b5b4e0f77c3a3810.pack
00007f73cec0c000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-937dae9140ac11d33831a697f9347f98baa25894.pack
00007f73d66d4000 125348K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d497de57bd0e623e5338eb553e0940ea2442274f.pack
00007f73de13d000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-188241ed7cb01328a4c32b09b0aff82311e32a1c.pack
00007f73e5c05000 125720K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5fc5649f65f8596a6b3dfd7c71d47d34e7b150e5.pack
00007f73ed6cb000 125720K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-aeb01e8db0b4d43181626d8b7a412e813a4a8757.pack
00007f73f5191000 120872K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fcebe7c0d63e44015ecc186ed74d45cb998c22ef.pack
00007f73fc79b000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-e84a2620505a10807e64727283f608b9661c39c0.pack
00007f7404263000 125720K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-56f5137267429f72a6e57ec70a637dc944cfeec7.pack
00007f740bd29000 125720K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-564815c1b986e15fa385daf51d6ecc9f44a07679.pack
00007f74137ef000 120868K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-4c28c08cbf0279f71e3c0a97cc91639fd6a8b308.pack
00007f741adf8000 125728K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-7e2d986a0faf6e464ff240b5bb21bb1b446b329f.pack
00007f74228c0000 484K rw--- [ anon ]
00007f7422939000 96840K r---- /usr/lib/locale/locale-archive
00007f74287cb000 8K r-x-- /lib64/libdl-2.12.so
00007f74287cd000 2048K ----- /lib64/libdl-2.12.so
00007f74289cd000 4K r---- /lib64/libdl-2.12.so
00007f74289ce000 4K rw--- /lib64/libdl-2.12.so
00007f74289cf000 1576K r-x-- /lib64/libc-2.12.so
00007f7428b59000 2048K ----- /lib64/libc-2.12.so
00007f7428d59000 16K r---- /lib64/libc-2.12.so
00007f7428d5d000 8K rw--- /lib64/libc-2.12.so
00007f7428d5f000 16K rw--- [ anon ]
00007f7428d63000 92K r-x-- /lib64/libpthread-2.12.so
00007f7428d7a000 2048K ----- /lib64/libpthread-2.12.so
00007f7428f7a000 4K r---- /lib64/libpthread-2.12.so
00007f7428f7b000 4K rw--- /lib64/libpthread-2.12.so
00007f7428f7c000 16K rw--- [ anon ]
00007f7428f80000 28K r-x-- /lib64/librt-2.12.so
00007f7428f87000 2044K ----- /lib64/librt-2.12.so
00007f7429186000 4K r---- /lib64/librt-2.12.so
00007f7429187000 4K rw--- /lib64/librt-2.12.so
00007f7429188000 1768K r-x-- /usr/lib64/libcrypto.so.1.0.1e
00007f7429342000 2044K ----- /usr/lib64/libcrypto.so.1.0.1e
00007f7429541000 108K r---- /usr/lib64/libcrypto.so.1.0.1e
00007f742955c000 48K rw--- /usr/lib64/libcrypto.so.1.0.1e
00007f7429568000 16K rw--- [ anon ]
00007f742956c000 84K r-x-- /lib64/libz.so.1.2.3
00007f7429581000 2044K ----- /lib64/libz.so.1.2.3
00007f7429780000 4K r---- /lib64/libz.so.1.2.3
00007f7429781000 4K rw--- /lib64/libz.so.1.2.3
00007f7429782000 128K r-x-- /lib64/ld-2.12.so
00007f74297a6000 464K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-004517110da3d0eaeea50b4eace06633eab4294c.pack
00007f742981a000 52K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-ee7d89bc7f8e5cd3e36c3655d967dfc8af3e6c7b.pack
00007f7429827000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-0381d5fab5a4f18e02eb18c3e12c5585702c9564.pack
00007f7429828000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-f26f972bc47c3a18cf47493e680c90fa3d5c9046.pack
00007f7429829000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-30fe739ba7e266ddf6f09a29fc2aa49eedd9e90b.pack
00007f742982a000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fb13617b92c223a9d017935c5ed27c9c8b0e62de.pack
00007f742982b000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d4bc55fff1083c9601bafeb49abf378cdc6f0727.pack
00007f742982c000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-e104e0bffca85b6346ef588c8c63d6ddadc567e3.pack
00007f742982d000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5eeddad5db2c0a44f33a3c41cd9216a7af0417df.pack
00007f742982e000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fc2614aa21f9284e0b76b6cc3ad9a90a0bae371e.pack
00007f742982f000 8K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-739869658456c4a7627279fceafa7c7c4c408acb.pack
00007f7429831000 20K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-feabd42e4392c52c8864a0397d10a73d7ada6fc8.pack
00007f7429836000 8K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-26462c3338a2e9302852788bad7b827a98748b65.pack
00007f7429838000 8K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-69b7829b456d813335b339545583ac9585ba9faa.pack
00007f742983a000 8K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-7e87b7b2f517ff2097026c55e98be6d1cec14587.pack
00007f742983c000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-eec773d189c951b5768eb85d33390d3d43db1d9f.pack
00007f742983d000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-25232ec34a92ab095486707f73fe5ad0a92bcb14.pack
00007f742983e000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5c633980ef7d381a4af2e7bf433c1c86154848c1.pack
00007f742983f000 20K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-2ec52679fc048e2db73cedc5bab30680f9b14bc9.pack
00007f7429875000 4K rw--- [ anon ]
00007f7429876000 400K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d18bf8e1b6a421fac697607e204da80b6c4c0370.pack
00007f74298da000 512K rw--- [ anon ]
00007f742995a000 44K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-004517110da3d0eaeea50b4eace06633eab4294c.idx
00007f7429965000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-4c28c08cbf0279f71e3c0a97cc91639fd6a8b308.idx
00007f7429966000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-ee7d89bc7f8e5cd3e36c3655d967dfc8af3e6c7b.idx
00007f7429967000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fcebe7c0d63e44015ecc186ed74d45cb998c22ef.idx
00007f7429968000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-0381d5fab5a4f18e02eb18c3e12c5585702c9564.idx
00007f7429969000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d497de57bd0e623e5338eb553e0940ea2442274f.idx
00007f742996a000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-f26f972bc47c3a18cf47493e680c90fa3d5c9046.idx
00007f742996b000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-33af25946a2fcb1c3e284b3093ac0eafa440cf46.idx
00007f742996c000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-30fe739ba7e266ddf6f09a29fc2aa49eedd9e90b.idx
00007f742996d000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-56f5137267429f72a6e57ec70a637dc944cfeec7.idx
00007f742996e000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fb13617b92c223a9d017935c5ed27c9c8b0e62de.idx
00007f742996f000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-aeb01e8db0b4d43181626d8b7a412e813a4a8757.idx
00007f7429970000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d4bc55fff1083c9601bafeb49abf378cdc6f0727.idx
00007f7429971000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-564815c1b986e15fa385daf51d6ecc9f44a07679.idx
00007f7429972000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-e104e0bffca85b6346ef588c8c63d6ddadc567e3.idx
00007f7429973000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5fc5649f65f8596a6b3dfd7c71d47d34e7b150e5.idx
00007f7429974000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5eeddad5db2c0a44f33a3c41cd9216a7af0417df.idx
00007f7429975000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-937dae9140ac11d33831a697f9347f98baa25894.idx
00007f7429976000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-fc2614aa21f9284e0b76b6cc3ad9a90a0bae371e.idx
00007f7429977000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-1468b8f7d29c999297b2411f9b94d86b0cd9965d.idx
00007f7429978000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-739869658456c4a7627279fceafa7c7c4c408acb.idx
00007f7429979000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-bb77ce8e83062bf2d26a3366b5b4e0f77c3a3810.idx
00007f742997a000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-feabd42e4392c52c8864a0397d10a73d7ada6fc8.idx
00007f742997b000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-dc37fbdaf5ad7f6c317a7c3d5ffecf53daa045b6.idx
00007f742997c000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-26462c3338a2e9302852788bad7b827a98748b65.idx
00007f742997d000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-3ecc274dd42f4783926224457225968b1fe1d95b.idx
00007f742997e000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-69b7829b456d813335b339545583ac9585ba9faa.idx
00007f742997f000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-7e2d986a0faf6e464ff240b5bb21bb1b446b329f.idx
00007f7429980000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-7e87b7b2f517ff2097026c55e98be6d1cec14587.idx
00007f7429981000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-bbef922979f633183f0bc207b5e52608b36d6434.idx
00007f7429982000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-5c633980ef7d381a4af2e7bf433c1c86154848c1.idx
00007f7429983000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-e84a2620505a10807e64727283f608b9661c39c0.idx
00007f7429984000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-eec773d189c951b5768eb85d33390d3d43db1d9f.idx
00007f7429985000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-c7d6e0b1e4da6a2aaa5eff3cad27f6fcbb38a5c9.idx
00007f7429986000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-25232ec34a92ab095486707f73fe5ad0a92bcb14.idx
00007f7429987000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-188241ed7cb01328a4c32b09b0aff82311e32a1c.idx
00007f7429988000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-be75b796793b738f7566a4be06491cfd3ad9c236.idx
00007f7429989000 24K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-d18bf8e1b6a421fac697607e204da80b6c4c0370.idx
00007f742998f000 20K rw--- [ anon ]
00007f7429998000 4K r---- /var/www/virtual/foo/bar/.git/objects/pack/pack-2ec52679fc048e2db73cedc5bab30680f9b14bc9.idx
00007f7429999000 28K r--s- /usr/lib64/gconv/gconv-modules.cache
00007f74299a0000 4K rw--- [ anon ]
00007f74299a1000 4K r---- /lib64/ld-2.12.so
00007f74299a2000 4K rw--- /lib64/ld-2.12.so
00007f74299a3000 4K rw--- [ anon ]
00007ffe76939000 84K rw--- [ stack ]
00007ffe7698b000 4K r-x-- [ anon ]
ffffffffff600000 4K r-x-- [ anon ]
total 2809104K
sh-4.1$ error: pack-objects died of signal 15
error: failed to run repack
推荐答案
也许是解决方案
我观察到，当我选择packSizeLimit很小时，这会产生相反的效果-它增加了.git目录而不是减小目录，这也可能导致达到位存储桶大小限制(未验证)
I observed, that when I choose the packSizeLimit very small, this is contra productive - it increased the .git directory instead of decreasing it, and this might cause also the bit bucket size limit to be reached (unverified)
1)将存储库重新同步到具有更多RAM的另一台计算机上，并将其正确打包
1) rsync the repo to another computer with more RAM and pack it properly
2)使用内存限制将其重新同步回服务器
2) rsync it back to server with the memory limit
3)增加到pack.packSizeLimit=300m
4)现在，在git gc之后，回购不会像疯了一样
4) Now after a git gc the repo does not grow like crazy
5)提交数据库转储
6)回购的大小增加了一倍
6) repo doubles its size
7)直接执行git gc
8)尺寸变好
9)现在按下
让我们看看几天后的情况...
let's see how it goes after some days ...
这篇关于Git gc占用大量内存，即使我限制了它的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
