howto：弹性豆串+部署码头+正常关机 [英] howto: elastic beanstalk + deploy docker + graceful shutdown

问题描述

在EB上托管一个docker容器，并在其上运行一个基于nodej的代码。
当我们重新部署我们的码头容器时，我们希望旧的关闭。

我找到了帮助&指导我们的代码如何接收'docker stop'命令产生的sigterm信号。

然而，进一步调查运行码头处理器的EB机器是：
/opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh
显示当从当前的翻转到新的分段容器时，旧的 docker kill'

有什么办法可以将此行为更改为停靠码头吗？

或者一般来说，推荐的方法来处理旧的容器？

谢谢！

解决方案

自我回答找到一个适用于我们的解决方案：

tl; dr：使用.ebextensions脚本在01flip之前运行脚本，您的脚本将确保停靠码头内的whatevers的优雅关闭发生

第一个，
你的应用程序（或者你在docker中运行的任何东西）必须能抓住一个信号，例如SIGINT，并正常关闭它。

这与Docker完全无关，您可以测试它运行在哪里（例如）
有很多关于获取信息的信息这种对网络上不同类型应用程序的行为（无论是ruby，node.js等...）

第二个
您的EB / Docker项目可以有一个.ebextensions文件夹，其中包含在部署时执行的所有类型的脚本。
我们把2个自定义脚本放入它，gracefulshutdown_01.config和gracefulshutdown_02.config文件看起来像这样：

  ＃gracefulshutdown_01.config 
命令：
 backup-original-flip-hook：
命令：cp -f /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh / opt / elasticbeanstalk / hooks / appdeploy / 01flip.sh.bak 
 test：'[！ -f /opt/elasticbeanstalk/hooks/appdeploy/01flip.sh.bak]'
 cleanup-custom-hooks：
命令：rm -f 05gracefulshutdown.sh 
 cwd：/ opt / elasticbeanstalk / hooks / appdeploy / enact 
 ignoreErrors：true 
  

和：

 ＃gracefulshutdown_02.config 
命令：
 reorder-original-flip-hook：
命令：mv / opt / springbeanstalk / hooks / appdeploy / enact / 01flip.sh /opt/elasticbeanstalk/hooks/appdeploy/enact/10flip.sh 
 test：'[-f /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh] '
 
文件：
/opt/elasticbeanstalk/hooks/appdeploy/enact/05gracefulshutdown.sh：
模式：000755
所有者：root 
组：root 
内容：| 
＃！/ bin / sh 
 
＃查找当前运行的docker 
 EB_CONFIG_DOCKER_CURRENT_APP_FILE = $（/ opt / elasticbeanstalk / bin / get-config container -k app_deploy_file）
 EB_CONFIG_DOCKER_CURRENT_APP =
 
如果[-f $ EB_CONFIG_DOCKER_CURRENT_APP_FILE];那么
 EB_CONFIG_DOCKER_CURRENT_APP =`cat $ EB_CONFIG_DOCKER_CURRENT_APP_FILE | cut -c 1-12` 
 echo应用程序容器上的正常关闭：$ EB_CONFIG_DOCKER_CURRENT_APP
 else 
 echo没有CURRENT APP到GRACEFUL SHUTDOWN FOUND
 exit 0 
 fi 
 
＃给所有正在运行的.js文件（不是stats !!）的graceful kill命令
 docker exec $ EB_CONFIG_DOCKER_CURRENT_APP sh -cps x -o pid，command | grep -E 'worker'| grep -v -E'forever | grep'| awk'{print $ 1}'| xargs docker exec $ EB_CONFIG_DOCKER_CURRENT_APP kill -s SIGINT 
 echosent kill signals
 
＃wait（最多5分钟），直到进程完成并终止自己
 TRIES = 100 
 until [$ TRIES -eq 0]; do 
 PIDS =`docker exec $ EB_CONFIG_DOCKER_CURRENT_APP sh -cps x -o pid，command | grep -E'workers'| grep -v -E'forever | grep'| awk'{print $ 1}'| cat` 
 echo TRIES $ TRIES PIDS $ PIDS 
 if [-z$ PIDS];然后
 echo完成停泊码头的优雅关闭$ EB_CONFIG_DOCKER_CURRENT_APP
 exit 0 
 else 
让TRIES- = 1 
睡3 
 fi 
完成
 
 echo无法正常关闭，请手动调查
 exit 1 
  

gracefulshutdown_01.config 是一个小的实用程序，可以备份原始的flip01并删除（如果存在）我们的自定义脚本。

gracefulshutdown_02.config 是魔术发生的地方。
它创建一个05gracefulshutdown颁发脚本，并确保翻转将会发生后，重命名为10flip。

05gracefulshutdown，自定义脚本，基本上这样做： p>

• 查找当前运行的docker


• 查找需要发送SIGINT的所有进程（对我们来说，它的进程在其名称中使用工作者


• 发送sigint到上述过程


• 循环：


• 检查如果以前的进程被杀死


• 如果尝试结束，继续循环尝试


• ，退出状态为1，不继续10flip，需要手动干扰。

这假设您只有1台停靠机在机器上运行，您可以手动跳为了检查出现错误的情况（因为我们从未发生过）。

我想象，它也可以在许多方面得到改善，所以很有趣。

Hi great people of stackoverflow,

Were hosting a docker container on EB with an nodejs based code running on it. When redeploying our docker container we'd like the old one to do a graceful shutdown.

I've found help & guides on how our code could receive a sigterm signal produced by 'docker stop' command.

However further investigation into the EB machine running docker at: /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh shows that when "flipping" from current to the new staged container, the old one is killed with 'docker kill'

Is there any way to change this behaviour to docker stop?
Or in general a recommended approach to handling graceful shutdown of the old container?

Thanks!

解决方案

Self answering as I've found a solution that works for us:

tl;dr: use .ebextensions scripts to run your script before 01flip, your script will make sure a graceful shutdown of whatevers inside the docker takes place

first, your app (or whatever your'e running in docker) has to be able to catch a signal, SIGINT for example, and shutdown gracefully upon it.
this is totally unrelated to Docker, you can test it running wherever (locally for example) There is a lot of info about getting this kind of behaviour done for different kind of apps on the net (be it ruby, node.js etc...)

Second, your EB/Docker based project can have a .ebextensions folder that holds all kinda of scripts to execute while deploying. we put 2 custom scripts into it, gracefulshutdown_01.config and gracefulshutdown_02.config file that looks something like this:

# gracefulshutdown_01.config
commands:
  backup-original-flip-hook:
    command: cp -f /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh /opt/elasticbeanstalk/hooks/appdeploy/01flip.sh.bak
    test: '[ ! -f /opt/elasticbeanstalk/hooks/appdeploy/01flip.sh.bak ]'
  cleanup-custom-hooks:
    command: rm -f 05gracefulshutdown.sh
    cwd: /opt/elasticbeanstalk/hooks/appdeploy/enact
    ignoreErrors: true

and:

# gracefulshutdown_02.config
commands:
  reorder-original-flip-hook:
    command: mv /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh /opt/elasticbeanstalk/hooks/appdeploy/enact/10flip.sh
    test: '[ -f /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh ]'

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/enact/05gracefulshutdown.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/bin/sh

      # find currently running docker
          EB_CONFIG_DOCKER_CURRENT_APP_FILE=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_file)
      EB_CONFIG_DOCKER_CURRENT_APP=""

      if [ -f $EB_CONFIG_DOCKER_CURRENT_APP_FILE ]; then
        EB_CONFIG_DOCKER_CURRENT_APP=`cat $EB_CONFIG_DOCKER_CURRENT_APP_FILE | cut -c 1-12`
        echo "Graceful shutdown on app container: $EB_CONFIG_DOCKER_CURRENT_APP"
      else
        echo "NO CURRENT APP TO GRACEFUL SHUTDOWN FOUND"
        exit 0
      fi

      # give graceful kill command to all running .js files (not stats!!)
      docker exec $EB_CONFIG_DOCKER_CURRENT_APP sh -c "ps x -o pid,command | grep -E 'workers' | grep -v -E 'forever|grep' " |  awk '{print $1}' | xargs docker exec $EB_CONFIG_DOCKER_CURRENT_APP kill -s SIGINT
      echo "sent kill signals"

      # wait (max 5 mins) until processes are done and terminate themselves
      TRIES=100
      until [ $TRIES -eq 0 ]; do
        PIDS=`docker exec $EB_CONFIG_DOCKER_CURRENT_APP sh -c "ps x -o pid,command | grep -E 'workers' | grep -v -E 'forever|grep' " | awk '{print $1}' | cat`
        echo TRIES $TRIES PIDS $PIDS
        if [ -z "$PIDS" ]; then
          echo "finished graceful shutdown of docker $EB_CONFIG_DOCKER_CURRENT_APP"
          exit 0
        else
          let TRIES-=1
          sleep 3
        fi
      done

      echo "failed to graceful shutdown, please investigate manually"
      exit 1

gracefulshutdown_01.config is a small util that backups the original flip01 and deletes (if exists) our custom script.

gracefulshutdown_02.config is where the magic happens. it creates a 05gracefulshutdown enact script and makes sure flip will happen afterwards by renaming it to 10flip.

05gracefulshutdown, the custom script, does this basically:

• find current running docker
• find all processes that need to be sent a SIGINT (for us its processes with 'workers' in its name
• send a sigint to the above processes
• loop:
• check if processes from before were killed
• continue looping for an amount of tries
• if tries are over, exit with status "1" and dont continue to 10flip, manual interference is needed.

this assumes you only have 1 docker running on the machine, and that you are able to manually hop on to check whats wrong in the case it fails (for us never happened yet).
I imagine it can also be improved in many ways, so have fun.

这篇关于howto：弹性豆串+部署码头+正常关机的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！