将S3存储桶作为文件系统挂载在AWS ECS容器上 [英] Mount S3 bucket as filesystem on AWS ECS container

问题描述

我正在尝试使用rexray/s3fs驱动程序将S3作为卷安装在AWS ECS docker容器上.

I am trying to mount S3 as a volume on AWS ECS docker container using rexray/s3fs driver.

我能够在安装插件的本地计算机上执行此操作

I am able to do this on my local machine, where I installed plugin

$docker plugin install rexray/s3fs

并将S3存储桶安装在docker容器上.

and mounted S3 bucket on docker container.

$docker plugin ls

ID                  NAME                 DESCRIPTION                                   ENABLED

3a0e14cadc17        rexray/s3fs:latest   REX-Ray FUSE Driver for Amazon Simple Storage   true 

$docker run -ti --volume-driver=rexray/s3fs -v s3-bucket:/data img

我正在尝试在AWS ECS上复制它.

I am trying replicate this on AWS ECS.

尝试遵循以下文档: https://docs.aws.amazon.com/AmazonECS/latest /developerguide/docker-volumes.html

Tried follow below document: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-volumes.html

如果我给Driver值，那么任务将无法运行并给出 由于没有容器实例满足其所有要求，因此无法下达任务."错误.

If I give Driver value then task is not able to run and giving "was unable to place a task because no container instance met all of its requirements." error.

我正在使用t2.medium实例并为其提供任务要求，因此它不应是硬件要求问题.

I am using t2.medium instance and giving of it requirement for task so it should not be H/W requirement issue.

如果我从作业定义任务中删除了驱动程序配置，则会执行该操作.

If I remove the Driver config from Job definition task gets executed.

似乎我很想配置一些东西.

It seems I am miss configuring something.

有人在尝试/尝试过相同的事情吗，请分享知识.

Is anyone trying/tried same thing, please share the knowledge.

谢谢！

推荐答案

您使用rexray/s3fs驱动程序的方法是正确的.

Your approach of using the rexray/s3fs driver is correct.

这些是我要在Amazon Linux 1上运行的步骤.

These are the steps I followed to get things working on Amazon Linux 1.

首先，您需要安装s3fs.

First you will need to install s3fs.

yum install -y gcc libstdc+-devel gcc-c+ fuse fuse-devel curl-devel libxml2-devel mailcap automake openssl-devel git gcc-c++
git clone https://github.com/s3fs-fuse/s3fs-fuse
cd s3fs-fuse/
./autogen.sh
./configure --prefix=/usr --with-openssl
make
make install

现在安装驱动程序.您可能需要在此处修改一些选项，例如使用IAM角色而不是访问密钥和AWS区域.

Now install the driver. There are some options here you might want to modify such as using an IAM role instead of Access Key and AWS region.

docker plugin install rexray/s3fs:latest S3FS_REGION=ap-southeast-2 S3FS_OPTIONS="allow_other,iam_role=auto,umask=000" LIBSTORAGE_INTEGRATION_VOLUME_OPERATIONS_MOUNT_ROOTPATH=/ --grant-all-permissions

现在是重新启动ECS代理程序非常重要的步骤.我也做了很好的更新.

Now the very important step of restarting the ECS agent. I also update for good measure.

yum update -y ecs-init
service docker restart && start ecs

您现在应该准备创建任务定义.重要的部分是您的卷配置，如下所示.

You should now be ready to create your task definition. The important part is your volume configuration which is shown below.

"volumes": [
  {
    "name": "name-of-your-s3-bucket",
    "host": null,
    "dockerVolumeConfiguration": {
      "autoprovision": false,
      "labels": null,
      "scope": "shared",
      "driver": "rexray/s3fs",
      "driverOpts": null
    }
  }
]

现在，您只需要在容器定义中指定安装点:

Now you just need to specify the mount point in the container definition:

"mountPoints": [
  {
    "readOnly": null,
    "containerPath": "/where/ever/you/want",
    "sourceVolume": "name-of-your-s3-bucket"
  }
]

现在，只要您具有访问s3存储桶的适当IAM权限，容器就应该启动，并且可以继续使用s3作为卷.

Now as long as you have appropriate IAM permissions for accessing the s3 bucket your container should start and you can get on with using s3 as a volume.

如果在执行"ATTRIBUTE"任务时遇到错误，请仔细检查插件是否已成功安装在ec2实例上，并且ecs代理已重新启动.还要仔细检查您的驱动程序名称是"rexray/s3fs".

If you get an error running the task that says "ATTRIBUTE" double check that the plugin has been successfully installed on the ec2 instance and the ecs agent has been restarted. Also double check your driver name is "rexray/s3fs".

这篇关于将S3存储桶作为文件系统挂载在AWS ECS容器上的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！