在多个Docker容器之间分发文件 [英] distribute files between multiple docker containers

问题描述

我们正在使用ECS部署多个容器(1000个)，每个容器使用相同的代码来启动应用程序，但是会消耗不同的配置文件(我们已经准备好文件.)您可以考虑以下情形:有1000个文件，有1000个容器，每个容器都会获得其唯一的文件并执行一些工作.

We are deploying multiple containers(1000) using ECS, each container use the same code to start an application, but will consume a different config file (we have the files ready.) You can think about the scenario like this: there is 1000 files, and there is 1000 containers, each container will get its unique file and do some work.

一种简单的方法是为每个容器实例创建单独的docker映像，唯一的区别是该配置文件.(绝对不是一个好方法.)

One simple approach would be create individual docker image for each container instance, with the only difference of that config file. (Definitely not a good approach.)

我们如何使用单个图像来做到这一点.ECS是否支持单个图像，但是您可以传递不同的文件/参数?

How can we do this using a single image. Does ECS support single image but you can pass in different files/parameters?

另一种方法是在容器启动时具有一些逻辑，它将从服务(如S3)获取文件.但是要求是每个容器都需要获得一个唯一的文件(1000个文件映射到1000个容器).最好的方法是什么?

Another approach would be have some logic when the container start, it will get the file from a service(like S3). But the requirement is that each container needs to get a unique file (1000 files maps to 1000 containers). What's the best way to do this?

推荐答案

使用1000个不同的配置(特别是1000个不同的配置)来配置1000个容器是不理想的.

Configuring 1000 containers specifically with 1000 different configs, even if it is the same container image, is not a desirable scenario.

您要么必须使业务流程管理工具知道哪些容器在使用哪些配置，这些容器仍在运行，而且要知道所有可用的配置.这不切实际，但是可行.

Either you will have to have the orchestration tool be aware of which containers are alive with which configs, and be aware of all the configs that are available. This is not practical, but doable.

否则容器本身将必须知道所有配置，并在它们之间就谁拥有什么配置进行通信.这是可行的，但很难做到.

Or the containers themselves will have to know of all the configs, and communicate between themselves about who's got what config. This is practical, but hard to do right.

第三个选择是构建一个托管配置的专用容器，其方式与容器的注册表相同.如果配置是文件，则它们可以简单地驻留在已安装的卷中，该卷由包含的配置注册表"应用程序监视.对于该应用程序而言，控制将哪些配置传递到哪些容器将是一件容易得多的任务.

A third option would be to build a specialized container that hosts the configs, in the same manner that you may have a registry for your containers. If the configs are files, they could simply reside in a mounted volume, that is monitored by the contained "config registry" application. It would be a much easier task for this application to control what configs have been passed to what containers.

您的应用程序必须在启动时向此配置注册表"注册，并接收其配置.如果与Docker集群API集成在一起，当列表中有尚未移交给应用程序容器的配置时，它可以控制扩展新容器.通过这种方式，您可以在配置注册表"容器的映射卷中删除新配置，这将导致新容器随配置一起旋转.

Your application would have to register with this "config registry" on startup, and receive its configuration. If integrated with the docker cluster API, it could control spinning up new containers when it had configs in its list, that were not handed off to an application container yet. In this manner, you could drop a new config in the mapped volume of the "config registry" container, and that would result in a new container being spun up with the configuration.

您还需要定期从注册表到每个容器进行运行状况检查.如果容器消失，则配置租约将结束"，它应设法为该配置创建一个新的容器.

You would also need a healthcheck from the registry to each container periodically. If a container vanishes, the configuration lease "ends" - and it should seek to create a new container for the configuration.

我确定这将需要一些工作-但结果将具有更大的可扩展性.如果您使用的是Java，请查看Spring Cloud Config-他们可能已经解决了这个用例，不确定.

This would require some work I am sure - but the result would be more scalable. In case you are using Java, look into Spring Cloud Config - they may have solved this usecase already, not sure.

嗯，这只是一个想法.听起来有用吗?

Well, that's just a thought. Does it sound useful?

这篇关于在多个Docker容器之间分发文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！