Docker容器的sshfs挂载冻结，但仅在通过Python挂载时冻结 [英] Docker container's sshfs mount freezes, but only when mounted by Python

问题描述

我有一台开发笔记本电脑( Mint 19.3 )和一台测试服务器( Ubuntu 18.04.4 LTS ).

笔记本电脑是Docker 版本19.03.5，内部版本633a0ea838 ，服务器是Docker 19.03.12版本，内部版本48a66213fe

我正在容器内运行Python 3.6代码，该容器使用 subprocess (下面的代码)在第三个服务器上创建sshfs挂载，然后python代码遍历挂载的目录./p>

在我的开发笔记本电脑上一切正常.但是在服务器上，该目录已挂载(并且可以通过 mount 命令看到)，但是进入目录的CD'只是挂起，而Python代码的后续 walk 挂起.(注意:python代码永远不会崩溃或出错.它只会永远挂起.)

但是，如果我在容器的命令行上手动使用相同的sshfs命令，则该目录可以正常工作.

我不知道如何解决此问题.

=== 2020-09-25 UPDATE ===

好.由于Python代码使用子进程，因此sshfs挂载显然可用于任何想要使用它的终端窗口.

我尝试从容器内的新终端窗口访问挂载，但是当我cd到挂载时-该窗口冻结了.

好吧，我把所有东西都放在一夜之间-现在当我尝试将CD装入安装架时...可以了.就像坐骑必须坐好几个小时才能工作.

有什么想法吗?

Python代码

  @ target_dir.setterdef target_dir(self，value):如果值为None:_tmp = tempfile.mkdtemp()self._TARGETDIR = _tmp别的:self._TARGETDIR =值def mount(自身):#sshfs username @ server-worker01:/test_project//mnt/test3 -o IdentityFile =/home/username/.ssh/sshprivkey，auto_cache，reconnect，transform_symlinks，follow_symlinks命令="sshfs {U} @ {S}:{RD} {LD}".format(U = self.user，S = self.server，RD = self.remote_dir，LD = self.target_dir)命令+ ="-o IdentityFile = {IDF}，自动缓存，重新连接，transform_symlinks，follow_symlinks，allow_other".format(IDF =键)cp = subprocess.getoutput(命令)如果cp！=":err =安装似乎有问题.({})" .format(cp)引发RuntimeError(err) 

在容器内部手动安装

  root @ 328100dd78be:/#mkdir/mnt/testroot @ 328100dd78be:/#sshfs用户名@ server-worker01:/test_project//mnt/test -o IdentityFile =/etc/ssl/certs/sshprivkey，auto_cache，reconnect，transform_symlinks，follow_symlinksroot @ 328100dd78be:/#cd/mnt/testroot @ 328100dd78be:/mnt/test#ls -la总共40drwxr-s --- 1 16543 41500 210八月6 07:07.drwxr-xr-x 1根根4096 Sep 24 14:27 ..drwxr-s --- 1 16543 41500 187 7月27日11:16 configFiles(...片段...)drwx--S --- 1 16543 41500 104 5月6日14:03提交 

但是，如果我让代码挂载它，则目录挂起

  root @ 328100dd78be:#ODCFFileCrawl.py2020-09-24 14:31:16,722-根-信息-记录仪已启动！与:{'loglevel':'10'}sshfs username @ server-worker01:/test_project//tmp/tmp0wwjcg_6 -o IdentityFile =/home/username/.ssh/sshprivkey，auto_cache，reconnect，transform_symlinks，follow_symlinks，allow_other，StrictHostKeyChecking = no2020-09-24 14:31:16,797-根-警告-将远程文件系统'server-worker01:/test_project/'安装到'/tmp/tmp7fig76gx'.(更改为新控制台)root @ 328100dd78be:/#安装覆盖在/类型覆盖上(rw，relatime，lowerdir =/var/lib/docker/overlay2/l/RWED5GN6GYRRGECDFN4LLIVVU7:/var/lib/docker/overlay2/l/7MTLBYSZ4ARQHKQ65TYBY5VSHW:/var/lib/docker/overlay2/L/EJX4L6Y:/var/lib/docker/overlay2/l/AFJWYKSAAP6E7RMIU3H2PUMRSN:/var/lib/docker/overlay2/l/CU3AWZFBMNMQLQSBUQSVCHOKWC:/var/lib/docker/overlay2/l/RBJZT34EXSVZD3UYORTLM6L5over//lib/OJ7YFP5Z4SEETA2UVO35HQ2K53:/var/lib/docker/overlay2/l/CBU6F6UI47VHQ2BW5NMGJE64UA:/var/lib/docker/overlay2/l/62ZEX3XIFI7VYTLYYJX6XNLJTM:/var/lib/lib/docker/over22/DLQ6/l/W5DJJWYP2VAYHTNDDHPY2KK2N3:/var/lib/docker/overlay2/l/ZOR2MOYW2NICOX3YVUUY6VXKVW:/var/lib/docker/overlay2/l/WLZEM73R55XKTPNVFCC5TRDCKTT/var/lib/docker/CM5:/overlay2/l/JDA4NCWU5RN5YNVC63USV4O3VC:/var/lib/docker/overlay2/l/CEGTXJOQH65FTIZLAJNTAPF24R:/var/lib/docker/overlay2/l/B4VV7H463RL3THLES637O57DUO:/var/lib/docker/Overlay2/L/KKRWT7BRZC2HRUKSB4C5:/var/lib/docker/overlay2/l/EOYAIHOSSFBSX7X5HEWWZQGXS4:/var/lib/docker/overlay2/l/T2ZFALNJHY37UVCYHXD6GC36R5:/var/lib/docker/overlay/Overlay/L/4YKYOA6升/PWZLNT3WAZPWPPYFOHZQ2SBHMG:在/var/lib中/搬运工/overlay2/升/W7UH56VKQDQH65GOWIGCYAMR3U，upperdir =的/var/lib中/搬运工/overlay2/9c2385aa1221d4fbcac321bfb7862dae23677bdf594ddea803315144b5124617/DIFF，WORKDIR =的/var/lib中/搬运工/overlay2/9c2385aa1221d4fbcac321bfb7862dae23677bdf594ddea803315144b5124617/工作)/proc类型proc上的proc(rw，nosuid，nodev，noexec，relatime)/dev类型的tmpfs上的tmpfs(rw，nosuid，size = 65536k，mode = 755)/dev/pts上的devpts类型devpts(rw，nosuid，noexec，relatime，gid = 5，mode = 620，ptmxmode = 666)/sys类型sysfs上的sysfs(ro，nosuid，nodev，noexec，relatime)/sys/fs/cgroup上的tmpfs类型tmpfs(ro，nosuid，nodev，noexec，relatime，mode = 755)/sys/fs/cgroup/systemd类型上的cgroup cgroup(ro，nosuid，nodev，noexec，relatime，xattr，name = systemd)/sys/fs/cgroup/freezer类型上的cgroup cgroup(ro，nosuid，nodev，noexec，relatime，freezer)/sys/fs/cgroup/cpu上的cgroup，cpuacct类型cgroup(ro，nosuid，nodev，noexec，relatime，cpu，cpuacct)/sys/fs/cgroup/hugetlb类型上的cgroup cgroup(ro，nosuid，nodev，noexec，relatime，hugetlb)/sys/fs/cgroup/内存类型上的cgroup cgroup(ro，nosuid，nodev，noexec，relatime，内存)/sys/fs/cgroup/perf_event上的cgroup类型cgroup(ro，nosuid，nodev，noexec，relatime，perf_event)/sys/fs/cgroup/blkio上的cgroup类型cgroup(ro，nosuid，nodev，noexec，relatime，blkio)/sys/fs/cgroup/pids上的cgroup类型cgroup(ro，nosuid，nodev，noexec，relatime，pids)/sys/fs/cgroup/net_cls上的cgroup，net_prio类型cgroup(ro，nosuid，nodev，noexec，relatime，net_cls，net_prio)/sys/fs/cgroup/cpuset上的cgroup类型cgroup(ro，nosuid，nodev，noexec，relatime，cpuset)/sys/fs/cgroup/rdma上的cgroup类型cgroup(ro，nosuid，nodev，noexec，relatime，rdma)/sys/fs/cgroup/devices上的cgroup类型cgroup(ro，nosuid，nodev，noexec，relatime，devices)/dev/mqueue上的mqueue类型mqueue(rw，nosuid，nodev，noexec，relatime)/etc/resolv.conf上的/dev/vda1类型ext4(rw，relatime，data = ordered)/etc/hostname上的/dev/vda1类型ext4(rw，relatime，data = ordered)/etc/hosts上的/dev/vda1类型ext4(rw，relatime，data = ordered)/dev/shm上的shm输入tmpfs(rw，nosuid，nodev，noexec，relatime，size = 65536k)/etc/ssl/certs上的/dev/vda1类型ext4(rw，relatime，data = ordered)/opt/vep/.vep类型ext4上的/dev/vdb1(rw，relatime，data = ordered)proc on/proc/bus类型proc(ro，relatime)/proc/fs类型proc上的proc(ro，relatime)/proc/irq类型proc上的proc(ro，relatime)/proc/sys类型proc上的proc(ro，relatime)proc/proc/sysrq触发类型proc(ro，relatime)/proc/acpi类型的tmpfs上的tmpfs(ro，relatime)/proc/kcore类型的tmpfs上的tmpfs(rw，nosuid，size = 65536k，mode = 755)/proc/keys上的tmpfs键入tmpfs(rw，nosuid，size = 65536k，mode = 755)/proc/timer_list上的tmpfs类型tmpfs(rw，nosuid，size = 65536k，mode = 755)/proc/sched_debug类型的tmpfs上的tmpfs(rw，nosuid，size = 65536k，mode = 755)/proc/scsi上的tmpfs类型tmpfs(ro，relatime)/sys/firmware上的tmpfs类型tmpfs(ro，relatime)username @ server-worker01:/test/project/在/mnt/测试类型fuse.sshfs(rw，nosuid，nodev，relatime，user_id = 0，group_id = 0)username @ server-worker01:/test_project/on/tmp/tmp7fig76gx类型fuse.sshfs(rw，nosuid，nodev，relatime，user_id = 0，group_id = 0，allow_other)root @ 328100dd78be:#cd/tmp/tmp7fig76gx(永远挂) 

=== 2020-12-04 @colidyre === 的更新

尝试一下...

mount.py

 导入临时文件导入子流程#target_dir =远程目录将被安装到的本地完整路径target_dir = tempfile.mkdtemp()user =<<您在远程服务器上的用户名>"server =<<远程服务器的服务器名称>".remote_dir =<<远程目录的完整路径>".key =<<远程服务器的ssh私钥的完整路径>".选项="-o IdentityFile = {IDF}，自动缓存，重新连接，transform_symlinks，follow_symlinks，allow_other" .format(IDF =键)命令="sshfs {U} @ {S}:{RD} {LD}".format(U =用户，S =服务器，RD = remote_dir，LD = target_dir)命令+ =选项cp = subprocess.getoutput(命令)如果cp！=":err =安装似乎有问题.({})" .format(cp)引发RuntimeError(err) 

Dockerfile的代码 注意:这只是来自 https://hub.docker的出厂VEP Docker容器.com/r/ensemblorg/ensembl-vep/，中间添加几行，将 mount.py 命令复制到/usr/bin 和入口点更改为 mount.py .参见...

  ##########################################mount.py的安装环境######################################### 

Dockerfile

  ###################################################阶段1-用于构建ensembl-vep的docker容器##################################################来自ubuntu:18.04作为生成器#更新aptitude并安装一些必需的软件包#Bio :: DB :: BigFile需要很多运行apt-get update&&&apt-get -y安装\建立必要的apt-utils \python3 \python3-pip \git \libpng-dev \zlib1g-dev \libbz2-dev \liblzma-dev \佩尔基于Perl的解压get#wget&&\#rm -rf/var/lib/apt/lists/*#设置VEP环境ENV OPT/opt/vepENV OPT_SRC $ OPT/srcENV HTSLIB_DIR $ OPT_SRC/htslibENV BRANCH版本/101#工作目录WORKDIR $ OPT_SRC#克隆/下载存储库/库如果[[$ BRANCH]=主人"；];\然后导出BRANCH_OPT ="；\否则，导出BRANCH_OPT =-b $ BRANCH"；\fi&&\#获取ensembl cpanfile以获取所需的Perl库的列表wget -q"https://raw.githubusercontent.com/Ensembl/ensembl/$BRANCH/cpanfile"-O"ensembl_cpanfile"；&&\#克隆ensembl-vep git存储库git clone $ BRANCH_OPT-深度1 https://github.com/Ensembl/ensembl-vep.git&&chmod u + x ensembl-vep/*.pl&&\#克隆ensembl-variation git存储库并编译C代码git clone $ BRANCH_OPT-深度1 https://github.com/Ensembl/ensembl-variation.git&&\mkdir var_c_code&&\cp ensembl-variation/C_code/*.c ensembl-variation/C_code/Makefile var_c_code/&&\rm -rf合奏变化&&\chmod u + x var_c_code/*&&\#克隆bioperl-ext git存储库-由Haplosaurus使用git clone --depth 1 https://github.com/bioperl/bioperl-ext.git&&\#下载ensembl-xs-它包含VEP中使用的某些关键子例程的编译版本wget https://github.com/Ensembl/ensembl-xs/archive/2.3.2.zip -O ensembl-xs.zip&&\解压缩-q ensembl-xs.zip&&mv ensembl-xs-2.3.2 ensembl-xs&&rm -rf ensembl-xs.zip&&\#克隆/下载其他存储库:需要bioperl-live，因此从ensembl-vep/cpanfile文件安装cpanm依赖项会占用较少的磁盘空间ensembl-vep/travisci/get_dependencies.sh&&\#仅保留生物活性的"Bio"生物.图书馆mv bioperl-live bioperl-live_bak&&mkdir bioperl-live&&mv bioperl-live_bak/Bio bioperl-live/&&rm -rf bioperl-live_bak&&\##对导入的库进行了大量清理，以减少docker镜像##rm -rf Bio-HTS/.??* Bio-HTS/更改Bio-HTS/免责声明Bio-HTS/MANIFEST * Bio-HTS/README Bio-HTS/脚本Bio-HTS/t Bio-HTS/travisci \bioperl-ext/.??* bioperl-ext/Bio/SeqIO bioperl-ext/Bio/Tools bioperl-ext/Makefile.PL bioperl-ext/README * bioperl-ext/t bioperl-ext/examples \ensembl-vep/.??* ensembl-vep/docker \ensembl-xs/.??* ensembl-xs/TODO ensembl-xs/更改ensembl-xs/安装ensembl-xs/清单ensembl-xs/README ensembl-xs/t ensembl-xs/travisci \htslib/.??* htslib/INSTALL htslib/NEWS htslib/README * htslib/test&&\#仅保留VEP所需的kent-335_base库-由Bio :: DB :: BigFile使用(bigWig解析)mv kent-335_base kent-335_base_bak&&mkdir -p kent-335_base/src&&\cp -R kent-335_base_bak/src/lib kent-335_base_bak/src/inc kent-335_base_bak/src/jkOwnLib kent-335_base/src/&&\cp kent-335_base_bak/src/*.sh kent-335_base/src/&&\rm -rf kent-335_base_bak#设置bioperl-extWORKDIR bioperl-ext/Bio/Ext/Align/RUN perl -pi -e"s |(cd libs.+)CFLAGS = \\\'| \ $ 1CFLAGS = \\\'-fPIC |"Makefile文件#安装htslib二进制文件(用于'bgzip'和'tabix')#htslib需要软件包'zlib1g-dev'，'libbz2-dev'和'liblzma-dev'WORKDIR $ HTSLIB_DIR运行make install&&rm -f Makefile * .c#编译变体LD C脚本WORKDIR $ OPT_SRC/var_c_code运行make&&rm -f Makefile * .c##################################################阶段2-用于构建ensembl-vep的docker容器##################################################来自Ubuntu:18.04#更新aptitude并安装一些必需的软件包#Bio :: DB :: BigFile需要很多运行apt-get update&&&apt-get -y安装\建立必要的python3 \python3-pip \git \盘根卷曲libmysqlclient-dev \libpng-dev \libssl-dev \zlib1g-dev \libbz2-dev \liblzma-dev \语言环境\openssl \佩尔基于Perl的解压sshfs \vim&&\apt-get -y清除手册页-dev&&\易于清洁\rm -rf/var/lib/apt/lists/*#必须在运行setup.py之前安装gitpython运行pip3安装gitpython##########################################mount.py的安装环境##########################################以本地FUSE conf为基础.这应该正确配置#与"user_allow_other"一起使用已启用复制mount.py/usr/bin/mount.pyRUN chown root:根/usr/bin/mount.py运行chmod 755/usr/bin/mount.py#在安装之前保留GIT下载运行mkdir/src运行chmod 777/src##########################################设置VEP环境#########################################ENV OPT/opt/vepENV OPT_SRC $ OPT/srcENV PERL5LIB_TMP $ PERL5LIB:$ OPT_SRC/ensp-vep:$ OPT_SRC/ensp-vep/modulesENV PERL5LIB $ PERL5LIB_TMP:$ OPT_SRC/bioperl-liveENV KENT_SRC $ OPT/src/kent-335_base/srcENV HTSLIB_DIR $ OPT_SRC/htslibENV MACHTYPE x86_64ENV DEPS $ OPT_SRCENV路径$ OPT_SRC/ensp-vep:$ OPT_SRC/var_c_code:$ PATHENV LANG_VAR en_US.UTF-8#创建vep用户运行useradd -r -m -U -d"$ OPT";-s/bin/bash -c"VEP用户"；-p''vep&&usermod -a -G sudo vep&&mkdir -p $ OPT_SRC使用者vep#将下载的库(阶段1)复制到该映像(阶段2)复制--chown = vep:vep --from = builder $ OPT_SRC $ OPT_SRC############################################################将用户更改为root用户，以进行以下编译/安装USER根#为haplo安装bioperl-ext，更快的比对(对C库的基于XS的BioPerl扩展)WORKDIR $ OPT_SRC/bioperl-ext/Bio/Ext/Align/运行perl Makefile.PL&&制作&&进行安装和安装rm -f Makefile *#安装ensembl-xs，使用某些Perl子例程在C中的重新实现来更快地运行WORKDIR $ OPT_SRC/ensembl-xs运行perl Makefile.PL&&制作&&进行安装和安装rm -f Makefile * cpanfileWORKDIR $ OPT_SRC#安装/编译更多库运行ensembl-vep/travisci/build_c.sh&&\#删除未使用的Bio-DB-HTS文件rm -rf Bio-HTS/cpanfile Bio-HTS/Build.PL Bio-HTS/Build Bio-HTS/_build Bio-HTS/INSTALL.pl&&\#安装ensembl perl依赖项(cpanm)cpanm --installdeps --with-recommends --notest --cpanfile ensembl_cpanfile.&&\cpanm --installdeps --with-recommends --notest --cpanfile ensembl-vep/cpanfile.&&\#安装cpanm后删除bioperl和cpanfiles，因为INSTALL.pl脚本将重新安装bioperlrm -rf bioperl-live ensembl_cpanfile ensembl-vep/cpanfile&&\#配置语言环境"，请参阅https://github.com/rocker-org/rocker/issues/19回声"$ LANG_VAR UTF-8";>>/etc/locale.gen&&locale-gen en_US.utf8&&\/usr/sbin/update-locale LANG = $ LANG_VAR&&\#复制htslib可执行文件.它还需要软件包"zlib1g-dev"，"libbz2-dev"和"liblzma-dev"cp $ HTSLIB_DIR/bgzip $ HTSLIB_DIR/tabix $ HTSLIB_DIR/htsfile/usr/local/bin/ENV LC_ALL $ LANG_VARENV LANG $ LANG_VAR#切换回vep用户使用者vepENV PERL5LIB $ PERL5LIB_TMP#VEP的最终步骤WORKDIR $ OPT_SRC/ensembl-vep#更新bash个人资料RUN回声>>$ OPT/.profile&&\echo PATH = $ PATH:\ $ PATH>>$ OPT/.profile&&\回声导出PATH>>$ OPT/.profile&&\#运行INSTALL.pl并删除ensemb-vep测试和travis./INSTALL.pl -a a -l -n&&rm -rf t travisci .travis.ymlENTRYPOINT ["mount.py"]#CMD tail -f/dev/null 

解决方案

我假设您想使用SSHFS将某些服务器的目录挂载到容器的文件系统中.您可以将该指令添加到Dockerfile中:

 从FROM版本:18.04运行apt-get update&&apt-get -y安装内容复制sshprivkey .ssh/sshprivkey运行mkdir/mnt/test运行sshfs username @ server-worker01:/test_project//mnt/test -o IdentityFile = .ssh/sshprivkey，auto_cache，reconnect，transform_symlinks，follow_symlinks 

您可以在此下面的Dockerfile中添加其他内容.您应该将 sshprivkey 文件添加到当前目录.

现在，如果您构建它:

  docker build -t your_desired_image_name. 

然后使用该图像运行一个容器，并检查其是否有效:

  docker run --privileged -it your_desired_image_nameroot @ container_id $ ls/mnt/test 

请记住添加该-privileged 标志，以使其正常工作.我从此处找到了.

I have a development laptop (Mint 19.3), and a test server (Ubuntu 18.04.4 LTS).

The laptop is Docker version 19.03.5, build 633a0ea838, the server is Docker version 19.03.12, build 48a66213fe

I'm running Python 3.6 code inside the container, which uses subprocess (code below) to create an sshfs mount to a third server, after which the python code walks through the mounted directory.

Everything works fine on my development laptop. But on the server, the directory mounts (and is seen with the mount command) however cd'ing into the directory just hangs, and the Python code's subsequent walk just hangs. (NOTE: The python code never crashes or errors out. It just hangs forever.)

HOWEVER, if I manually use the same sshfs command at the container's command line, the directory works fine.

I'm at a loss as to how to troubleshoot this.

===2020-09-25 UPDATE===

OK. Since the Python code uses subprocess, the sshfs mount is obviously available to any terminal windows that wants to use it.

I have tried accessing the mount from a new terminal window inside the container, but when I cd to the mount - the window just freezes.

Well, I left everything sitting overnight - and now when I try to cd into the mount ... it works. It's like the mount has to sit for hours before it will work.

Any ideas?

Python code

@target_dir.setter    
def target_dir(self, value):
    if value is None:
        _tmp = tempfile.mkdtemp()
        self._TARGETDIR = _tmp

    else:
        self._TARGETDIR = value


def mount(self):
    # sshfs username@server-worker01:/test_project/ /mnt/test3 -o IdentityFile=/home/username/.ssh/sshprivkey,auto_cache,reconnect,transform_symlinks,follow_symlinks
    command = "sshfs {U}@{S}:{RD} {LD}".format(
        U  = self.user, 
        S  = self.server, 
        RD = self.remote_dir, 
        LD = self.target_dir)
    command += " -o IdentityFile={IDF},auto_cache,reconnect,transform_symlinks,follow_symlinks,allow_other".format(IDF = key)

    cp = subprocess.getoutput(command)
    if cp != "":
        err = "Something appears to be wrong with the mount. ({})".format(cp)
        raise RuntimeError(err) 

Mounting manually inside container works

root@328100dd78be:/# mkdir /mnt/test
root@328100dd78be:/# sshfs username@server-worker01:/test_project/ /mnt/test -o IdentityFile=/etc/ssl/certs/sshprivkey,auto_cache,reconnect,transform_symlinks,follow_symlinks
root@328100dd78be:/# cd /mnt/test
root@328100dd78be:/mnt/test# ls -la
total 40
drwxr-s--- 1 16543 41500  210 Aug  6 07:07 .
drwxr-xr-x 1 root  root  4096 Sep 24 14:27 ..
drwxr-s--- 1 16543 41500  187 Jul 27 11:16 configFiles
(...snip...)
drwx--S--- 1 16543 41500  104 May  6 14:03 submission

But if I let the code mount it, the dir hangs

root@328100dd78be:# ODCFFileCrawl.py 
2020-09-24 14:31:16,722 - root - INFO - logger started! with:{'loglevel': '10'}
sshfs username@server-worker01:/test_project/ /tmp/tmp0wwjcg_6 -o IdentityFile=/home/username/.ssh/sshprivkey,auto_cache,reconnect,transform_symlinks,follow_symlinks,allow_other,StrictHostKeyChecking=no

2020-09-24 14:31:16,797 - root - WARNING - Mounted remote filesystem 'server-worker01:/test_project/' to '/tmp/tmp7fig76gx'.

(change to new console)
root@328100dd78be:/# mount
overlay on / type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay2/l/RWED5GN6GYRRGECDFN4LLIVVU7:/var/lib/docker/overlay2/l/7MTLBYSZ4ARQHKQ65TYBY5VSHW:/var/lib/docker/overlay2/l/EJX3L6YHRYB5IICXKAIH4PHT5I:/var/lib/docker/overlay2/l/AFJWYKSAAP6E7RMIU3H2PUMRSN:/var/lib/docker/overlay2/l/CU3AWZFBMNMQLQSBUQSVCHOKWC:/var/lib/docker/overlay2/l/RBJZT34EXSVZD3UYORTLM6L5UX:/var/lib/docker/overlay2/l/OJ7YFP5Z4SEETA2UVO35HQ2K53:/var/lib/docker/overlay2/l/CBU6F6UI47VHQ2BW5NMGJE64UA:/var/lib/docker/overlay2/l/62ZEX3XIFI7VYTLYYJX6XNLJTM:/var/lib/docker/overlay2/l/NBRXSW53NRLL5KRGYQ2BDL2ICZ:/var/lib/docker/overlay2/l/W5DJJWYP2VAYHTNDDHPY2KK2N3:/var/lib/docker/overlay2/l/ZOR2MOYW2NICOX3YVUUY6VXKVW:/var/lib/docker/overlay2/l/WLZEM73R55XKTPNVFCC5TRDCKT:/var/lib/docker/overlay2/l/PARGLNJTE4CM7PTTS5UOLXAIOC:/var/lib/docker/overlay2/l/JDA4NCWU5RN5YNVC63USV4O3VC:/var/lib/docker/overlay2/l/CEGTXJOQH65FTIZLAJNTAPF24R:/var/lib/docker/overlay2/l/B4VV7H463RL3THLES637O57DUO:/var/lib/docker/overlay2/l/KCENZYGKRWT7BRZC2HRUKSB4C5:/var/lib/docker/overlay2/l/EOYAIHOSSFBSX7X5HEWWZQGXS4:/var/lib/docker/overlay2/l/T2ZFALNJHY37UVCYHXD6GC36R5:/var/lib/docker/overlay2/l/4ACYIA6XBKYGFV5BUHYWTOALAY:/var/lib/docker/overlay2/l/PWZLNT3WAZPWPPYFOHZQ2SBHMG:/var/lib/docker/overlay2/l/W7UH56VKQDQH65GOWIGCYAMR3U,upperdir=/var/lib/docker/overlay2/9c2385aa1221d4fbcac321bfb7862dae23677bdf594ddea803315144b5124617/diff,workdir=/var/lib/docker/overlay2/9c2385aa1221d4fbcac321bfb7862dae23677bdf594ddea803315144b5124617/work)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev type tmpfs (rw,nosuid,size=65536k,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666)
sysfs on /sys type sysfs (ro,nosuid,nodev,noexec,relatime)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,relatime,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (ro,nosuid,nodev,noexec,relatime,xattr,name=systemd)
cgroup on /sys/fs/cgroup/freezer type cgroup (ro,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (ro,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (ro,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/memory type cgroup (ro,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/perf_event type cgroup (ro,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/blkio type cgroup (ro,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/pids type cgroup (ro,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (ro,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/cpuset type cgroup (ro,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/rdma type cgroup (ro,nosuid,nodev,noexec,relatime,rdma)
cgroup on /sys/fs/cgroup/devices type cgroup (ro,nosuid,nodev,noexec,relatime,devices)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
/dev/vda1 on /etc/resolv.conf type ext4 (rw,relatime,data=ordered)
/dev/vda1 on /etc/hostname type ext4 (rw,relatime,data=ordered)
/dev/vda1 on /etc/hosts type ext4 (rw,relatime,data=ordered)
shm on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=65536k)
/dev/vda1 on /etc/ssl/certs type ext4 (rw,relatime,data=ordered)
/dev/vdb1 on /opt/vep/.vep type ext4 (rw,relatime,data=ordered)
proc on /proc/bus type proc (ro,relatime)
proc on /proc/fs type proc (ro,relatime)
proc on /proc/irq type proc (ro,relatime)
proc on /proc/sys type proc (ro,relatime)
proc on /proc/sysrq-trigger type proc (ro,relatime)
tmpfs on /proc/acpi type tmpfs (ro,relatime)
tmpfs on /proc/kcore type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /proc/keys type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /proc/timer_list type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /proc/sched_debug type tmpfs (rw,nosuid,size=65536k,mode=755)
tmpfs on /proc/scsi type tmpfs (ro,relatime)
tmpfs on /sys/firmware type tmpfs (ro,relatime)
username@server-worker01:/test_project/ on /mnt/test type fuse.sshfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0)
username@server-worker01:/test_project/ on /tmp/tmp7fig76gx type fuse.sshfs (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)

root@328100dd78be:# cd /tmp/tmp7fig76gx
(hangs forever)

===2020-12-04 UPDATE FOR @colidyre===

Try this...

mount.py

import tempfile
import subprocess

# target_dir = The local full path to which the remote directory will be mounted
target_dir = tempfile.mkdtemp() 
user = "<Your username on the remote server >"
server = "<The server name for the remote server>"
remote_dir = "<Full path of the remote directory>"
key = "<Full path of the ssh private key for the remote server>"
options = " -o IdentityFile={IDF},auto_cache,reconnect,transform_symlinks,follow_symlinks,allow_other".format(IDF = key)

command = "sshfs {U}@{S}:{RD} {LD}".format(
    U  = user, 
    S  = server, 
    RD = remote_dir, 
    LD = target_dir)
command += options
cp = subprocess.getoutput(command)
if cp != "":
    err = "Something appears to be wrong with the mount. ({})".format(cp)
    raise RuntimeError(err) 

Code for Dockerfile NOTE: This is just the factory VEP Docker container from https://hub.docker.com/r/ensemblorg/ensembl-vep/ with lines added in the middle to copy the mount.py command into /usr/bin, and the entrypoint chnaged to mount.py. See section ...

##########################################
# Setup environment for mount.py
##########################################

Dockerfile

###################################################
# Stage 1 - docker container to build ensembl-vep #
###################################################
FROM ubuntu:18.04 as builder

# Update aptitude and install some required packages
# a lot of them are required for Bio::DB::BigFile
RUN apt-get update && apt-get -y install \
    build-essential \
    apt-utils \
      python3 \
      python3-pip \
    git \
    libpng-dev \
    zlib1g-dev \
    libbz2-dev \
    liblzma-dev \
    perl \
    perl-base \
    unzip \
    wget
#    wget && \
#    rm -rf /var/lib/apt/lists/*

# Setup VEP environment
ENV OPT /opt/vep
ENV OPT_SRC $OPT/src
ENV HTSLIB_DIR $OPT_SRC/htslib
ENV BRANCH release/101

# Working directory
WORKDIR $OPT_SRC

# Clone/download repositories/libraries
RUN if [ "$BRANCH" = "master" ]; \
    then export BRANCH_OPT=""; \
    else export BRANCH_OPT="-b $BRANCH"; \
    fi && \
    # Get ensembl cpanfile in order to get the list of the required Perl libraries
    wget -q "https://raw.githubusercontent.com/Ensembl/ensembl/$BRANCH/cpanfile" -O "ensembl_cpanfile" && \
    # Clone ensembl-vep git repository
    git clone $BRANCH_OPT --depth 1 https://github.com/Ensembl/ensembl-vep.git && chmod u+x ensembl-vep/*.pl && \
    # Clone ensembl-variation git repository and compile C code
    git clone $BRANCH_OPT --depth 1 https://github.com/Ensembl/ensembl-variation.git && \
    mkdir var_c_code && \
    cp ensembl-variation/C_code/*.c ensembl-variation/C_code/Makefile var_c_code/ && \
    rm -rf ensembl-variation && \
    chmod u+x var_c_code/* && \
    # Clone bioperl-ext git repository - used by Haplosaurus
    git clone --depth 1 https://github.com/bioperl/bioperl-ext.git && \
    # Download ensembl-xs - it contains compiled versions of certain key subroutines used in VEP
    wget https://github.com/Ensembl/ensembl-xs/archive/2.3.2.zip -O ensembl-xs.zip && \
    unzip -q ensembl-xs.zip && mv ensembl-xs-2.3.2 ensembl-xs && rm -rf ensembl-xs.zip && \
    # Clone/Download other repositories: bioperl-live is needed so the cpanm dependencies installation from the ensembl-vep/cpanfile file takes less disk space
    ensembl-vep/travisci/get_dependencies.sh && \
    # Only keep the bioperl-live "Bio" library
    mv bioperl-live bioperl-live_bak && mkdir bioperl-live && mv bioperl-live_bak/Bio bioperl-live/ && rm -rf bioperl-live_bak && \
    ## A lot of cleanup on the imported libraries, in order to reduce the docker image ##
    rm -rf Bio-HTS/.??* Bio-HTS/Changes Bio-HTS/DISCLAIMER Bio-HTS/MANIFEST* Bio-HTS/README Bio-HTS/scripts Bio-HTS/t Bio-HTS/travisci \
           bioperl-ext/.??* bioperl-ext/Bio/SeqIO bioperl-ext/Bio/Tools bioperl-ext/Makefile.PL bioperl-ext/README* bioperl-ext/t bioperl-ext/examples \
           ensembl-vep/.??* ensembl-vep/docker \
           ensembl-xs/.??* ensembl-xs/TODO ensembl-xs/Changes ensembl-xs/INSTALL ensembl-xs/MANIFEST ensembl-xs/README ensembl-xs/t ensembl-xs/travisci \
           htslib/.??* htslib/INSTALL htslib/NEWS htslib/README* htslib/test && \
    # Only keep needed kent-335_base libraries for VEP - used by Bio::DB::BigFile (bigWig parsing)
    mv kent-335_base kent-335_base_bak && mkdir -p kent-335_base/src && \
    cp -R kent-335_base_bak/src/lib kent-335_base_bak/src/inc kent-335_base_bak/src/jkOwnLib kent-335_base/src/ && \
    cp kent-335_base_bak/src/*.sh kent-335_base/src/ && \
    rm -rf kent-335_base_bak

# Setup bioperl-ext
WORKDIR bioperl-ext/Bio/Ext/Align/
RUN perl -pi -e"s|(cd libs.+)CFLAGS=\\\'|\$1CFLAGS=\\\'-fPIC |" Makefile.PL

# Install htslib binaries (for 'bgzip' and 'tabix')
# htslib requires the packages 'zlib1g-dev', 'libbz2-dev' and 'liblzma-dev'
WORKDIR $HTSLIB_DIR
RUN make install && rm -f Makefile *.c

# Compile Variation LD C scripts
WORKDIR $OPT_SRC/var_c_code
RUN make && rm -f Makefile *.c


###################################################
# Stage 2 - docker container to build ensembl-vep #
###################################################
FROM ubuntu:18.04

# Update aptitude and install some required packages
# a lot of them are required for Bio::DB::BigFile
RUN apt-get update && apt-get -y install \
    build-essential \
    python3 \
    python3-pip \
    git \
    cpanminus \
    curl \
    libmysqlclient-dev \
    libpng-dev \
    libssl-dev \
    zlib1g-dev \
    libbz2-dev \
    liblzma-dev \
    locales \
    openssl \
    perl \
    perl-base \
    unzip \
    sshfs \
    vim && \
    apt-get -y purge manpages-dev && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# gitpython must be installed before setup.py is run
RUN pip3 install gitpython

##########################################
# Setup environment for mount.py
##########################################

# Takes the local FUSE conf as the base. This should be properly configured
#  with "user_allow_other" enabled
COPY mount.py /usr/bin/mount.py
RUN chown root:root /usr/bin/mount.py
RUN chmod 755 /usr/bin/mount.py

# To hold the GIT downloads prior to install
RUN mkdir /src
RUN chmod 777 /src

##########################################
# Setup VEP environment
##########################################

ENV OPT /opt/vep
ENV OPT_SRC $OPT/src
ENV PERL5LIB_TMP $PERL5LIB:$OPT_SRC/ensembl-vep:$OPT_SRC/ensembl-vep/modules
ENV PERL5LIB $PERL5LIB_TMP:$OPT_SRC/bioperl-live
ENV KENT_SRC $OPT/src/kent-335_base/src
ENV HTSLIB_DIR $OPT_SRC/htslib
ENV MACHTYPE x86_64
ENV DEPS $OPT_SRC
ENV PATH $OPT_SRC/ensembl-vep:$OPT_SRC/var_c_code:$PATH
ENV LANG_VAR en_US.UTF-8

# Create vep user
RUN useradd -r -m -U -d "$OPT" -s /bin/bash -c "VEP User" -p '' vep && usermod -a -G sudo vep && mkdir -p $OPT_SRC
USER vep

# Copy downloaded libraries (stage 1) to this image (stage 2)
COPY --chown=vep:vep --from=builder $OPT_SRC $OPT_SRC
#############################################################

# Change user to root for the following complilations/installations
USER root

# Install bioperl-ext, faster alignments for haplo (XS-based BioPerl extensions to C libraries)
WORKDIR $OPT_SRC/bioperl-ext/Bio/Ext/Align/
RUN perl Makefile.PL && make && make install && rm -f Makefile*

# Install ensembl-xs, faster run using re-implementation in C of some of the Perl subroutines
WORKDIR $OPT_SRC/ensembl-xs
RUN perl Makefile.PL && make && make install && rm -f Makefile* cpanfile

WORKDIR $OPT_SRC
# Install/compile more libraries
RUN ensembl-vep/travisci/build_c.sh && \
    # Remove unused Bio-DB-HTS files
    rm -rf Bio-HTS/cpanfile Bio-HTS/Build.PL Bio-HTS/Build Bio-HTS/_build Bio-HTS/INSTALL.pl && \
    # Install ensembl perl dependencies (cpanm)
    cpanm --installdeps --with-recommends --notest --cpanfile ensembl_cpanfile . && \
    cpanm --installdeps --with-recommends --notest --cpanfile ensembl-vep/cpanfile . && \
    # Delete bioperl and cpanfiles after the cpanm installs as bioperl will be reinstalled by the INSTALL.pl script
    rm -rf bioperl-live ensembl_cpanfile ensembl-vep/cpanfile && \
    # Configure "locale", see https://github.com/rocker-org/rocker/issues/19
    echo "$LANG_VAR UTF-8" >> /etc/locale.gen && locale-gen en_US.utf8 && \
    /usr/sbin/update-locale LANG=$LANG_VAR && \
    # Copy htslib executables. It also requires the packages 'zlib1g-dev', 'libbz2-dev' and 'liblzma-dev'
    cp $HTSLIB_DIR/bgzip $HTSLIB_DIR/tabix $HTSLIB_DIR/htsfile /usr/local/bin/

ENV LC_ALL $LANG_VAR
ENV LANG $LANG_VAR

# Switch back to vep user
USER vep
ENV PERL5LIB $PERL5LIB_TMP

# Final steps for VEP
WORKDIR $OPT_SRC/ensembl-vep
# Update bash profile
RUN echo >> $OPT/.profile && \
    echo PATH=$PATH:\$PATH >> $OPT/.profile && \
    echo export PATH >> $OPT/.profile && \
    # Run INSTALL.pl and remove the ensemb-vep tests and travis
    ./INSTALL.pl -a a -l -n && rm -rf t travisci .travis.yml

ENTRYPOINT ["mount.py"]
# CMD tail -f /dev/null

解决方案

I am assuming you want to mount some server's directory to container's filesystem using SSHFS. You could add that instruction to the Dockerfile:

FROM ubuntu:18.04
RUN apt-get update && apt-get -y install stuff
COPY sshprivkey .ssh/sshprivkey
RUN mkdir /mnt/test
RUN sshfs username@server-worker01:/test_project/ /mnt/test -o IdentityFile=.ssh/sshprivkey,auto_cache,reconnect,transform_symlinks,follow_symlinks

You can add other stuff that you have in your Dockerfile below this. You should add sshprivkey file to your current directory.

Now if you build it:

docker build -t your_desired_image_name .

And then run a container using that image and check whether it worked:

docker run --privileged -it your_desired_image_name 

root@container_id$ ls /mnt/test

Remember to add that --privileged flag for it to work. I found it out from here.

这篇关于Docker容器的sshfs挂载冻结，但仅在通过Python挂载时冻结的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！