将存储库从Github迁移到自托管的Gitlab时更新和修复LFS [英] Updating and repairing LFS when migrating repositories from Github to selfhosted Gitlab

问题描述

对于我的公司，我必须将所有公司存储库从Github迁移到我们自己托管的Gitlab-Server.

For my company I have to migrate all our company repositories from Github to our self hosted Gitlab-Server.

我遇到的主要问题是，对于大多数存储库，我们都使用 Git-LFS

The main problem I have is that we use Git-LFS for the most of our repositories.

因此，在克隆存储库时，我必须通过运行来修复Git-LFS

Therefore when cloning a repository I have to repair the Git-LFS by running

git lfs fetch --all
git lfs push remoteOnGitlab

我正在为此编写脚本(由于其中包含大量存储库)，因此使用Github API列出了所有存储库的列表:

I'm writing on a script for that - since it are a lot of repositories - using the Github API to work off a list of all our repositories:

#!/bin/bash

###_Adjust those_#######################################################
githubUrlStart="https://"
githubLink="github.com/<orgaName>"
githubApiLink="api.github.com/orgs/<orgaName>/repos"

# maxRepoCount is needed if the organisation has more than 30 repos
# otherwise only the first 30 repos would be mirrored
maxRepoCount="200"

gitlabUrlStart="http://"
gitlabLink="<gitlabURL>/<groupName>"
########################################################################


###_Get Sensible Data at runtime_###
bold=$(tput bold)
normal=$(tput sgr0)
                                           
echo -n "${bold}Github Username: ${normal}"
read -e githubName
echo -n "${bold}Github Password: ${normal}"
read -e -s githubPassword
echo ""
echo -n "${bold}Gitlab Username: ${normal}"
read -e gitlabName
echo -n "${bold}Gitlab Password: ${normal}"
read -e -s gitlabPassword

###_Add folder for cloned git repos_###
mkdir gits

###_GET JSON of all REPOS from the Github API_###
# writes output to file test for later use
curl "$githubUrlStart${githubName//\"}:${githubPassword//\"}@${githubApiLink//\"}?per_page=${maxRepoCount//\"}" > test 

###_Get URLS and NAME_###
#  -> http://xmodulo.com/how-to-parse-json-string-via-command-line-on-linux.html #
NAMES=$(cat test | jq ".[] | .name");

###_Transform into Array_###
NAMES_ARRAY=()

COUNT=0
for name in $NAMES
do
    NAMES_ARRAY[$COUNT]=$name
                             
    COUNT=$((COUNT+1))
done


###_Clone each repo local and use it to repair git lfs_###                                                                                            
COUNT=0                                                                                       
for reponame in "${NAMES_ARRAY[@]}"                                                           
do                                                                                            
    ###_Build full link for cloning_###
    fulllink="$githubUrlStart$githubName:$githubPassword@$githubLink/${reponame//\"}"     
                                                                                              
    ###_Clone from github using git lfs_###
    git lfs clone "${fulllink//\"}" "gits/${reponame//\"}"
                                                                                              
    ###_Enter folder of cloned repository_###                                                   
    cd "gits/${reponame//\"}"
                                                                                              
    ###_repair git lfs_###
    # get lfs from github
    git lfs fetch --all
    
    # add the remote for gitlab
    gitlabRemote="$gitlabUrlStart$gitlabName:$gitlabPassword@$gitlabLink/${reponame//\"}.git" 
    git remote add gitlab "${gitlabRemote//\"}"
    
    # Just for security remove the github remote                                                  
    git remote remove origin                                                                  
    
    # Push lfs to the gitlab remote
    git lfs push gitlab --all                                                                 
       
    ##_leave folder!_##                                                                       
    cd ../..          #                                                                       
   
    ##_remove folder to save disk space_##                                                    
    rm -rf gits/${reponame//\"}
                                                                                              
    COUNT=$((COUNT+1))
done

###_Remove the gits folder_####
rm -rf gits

我的问题在这里:
看来这只能正常工作一次，但是如果我再次在其上运行脚本，则存储库不会更新为最新状态.另外我也不确定，但我想它到目前为止只能在master分支上工作.

My problem here:
It seems this works fine only once, but the repositories get not updated to the newest state if I run the script over them again. Also I'm not sure but I guess it will only work on the master branch so far.

为了始终从gitlab上始终从github获取git repo的最新状态，我需要更改什么?(运行后，我在Gitlab上仍具有2个月前的状态...)

What do I have to change in order to allways get the newest state of the git repo from github also on gitlab? (After running it I still have the state of from 2 months ago on Gitlab...)

我还看到了此处的另一种解决方案，该解决方案显然可以解决多分支问题:

I also have seen another solution here which apparently would handle the multibranch problem:

git clone --mirror path/to/original path/to/dest/.git
cd path/to/dest
git config --bool core.bare false
git checkout anybranch

我可能也可以结合

git push --mirrow path/to/otherRemote

如评论中所述.

这当然会更简单..但是:
这与LFS兼容吗?例如，

This would ofcourse be way simplier .. BUT:
Is this compatible with LFS? let's say e.g. as

git lfs clone --mirrow /originOnGithub /localFolder
cd /localFolder
git lfs fetch --all
git lfs push --mirrow /remoteOnGitlab

推荐答案

答案是:是的！

但是我有一个非常愚蠢的错字:当然是 --mirror 而不是--mirrow

But I had a very stupid typo: ofcourse it is --mirror not --mirrow

现在为每个项目运行一次

So now for each project a run

git lfs clone --mirror /originOnGithub /localFolder
cd /localFolder
git lfs fetch --all
git lfs push --mirror /remoteOnGitlab

并修复LFS文件，并更新Gitlab服务器上存储库中的所有分支.

and it repairs the LFS files AND updates all branches in the repository on the Gitlab-Server.

这篇关于将存储库从Github迁移到自托管的Gitlab时更新和修复LFS的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！