在Mesos上的Docker：Volume放在哪个节点上？ [英] Docker on Mesos: Volume is placed on which node?

问题描述

我将设置一个Mesos群集来运行一次性使用的码头作业，例如长时间的计算。当然，我想得到计算结果，所以我想我应该使用Docker卷。

I will be setting up a Mesos cluster to run single-use docker jobs, e.g. long rapidminer computations. Of course I want to get the result of the computation, so I think I should use Docker volumes for that.

现在，当我发送一个docker作业到一个集群时，指定例如在Marathon或Chronos的JSON作业文件中的卷，我的计算结果在哪里？

Now, when I send a docker job to a cluster, specifying the volume for example in a JSON job file for Marathon or Chronos, where does the result of my computation land?

我猜到它被放入从节点上的相应目录，但我真的要进入Mesos界面，查看哪个节点执行了我的工作，ssh进入该节点并复制我的结果文件？
这似乎非常违反了从单台计算机抽取Mesos的全部想法。

I am guessing that it is put into the respective directory on the slave node, but do I really have to go into the Mesos interface, look up which node executed my job, ssh into that node and copy my resulting file out? This seems very counterintuitive to the whole idea of Mesos of abstracting from single computers.

这种场景的优雅解决方案是什么？我对集群管理非常新颖，所以我可以想到的唯一好的解决方案是分布式文件系统，尽管我不知道这是否会在Marathon或Chronos的工作文件中得到支持。

What would be the elegant solution for this scenario? I am very new to cluster management, so the only good solution I could think of was a distributed filesystem, although I don't know if this would be supported in the jobfile of Marathon or Chronos.

推荐答案

可以肯定的是，Mesos假定所有的最终数据都存储在任务完成的某个地方，这是你的，或者如果你想要的话，你的任务或你的框架有责任确保这一点。如果要保留中间结果，或者在任务之间共享结果，可以查看持续的卷，这些卷正在开发中，并希望在下一个Mesos发行版中。请注意，它们被认为是节点资源的一部分，并且不被复制，因此在节点故障的情况下将丢失。

It is safe to say that Mesos assumes that all your final data is stored somewhere when you task finishes, and it's your, or if you want, your task's or your framework's responsibility to ensure this. If you want to persist intermediate results, or share results between tasks, you can look at persistent volumes, which are currently under development and will—hopefully—land in the next Mesos release. Be advised, that they are considered part of node resources and are not replicated, hence will be lost in case of node failure.

作为分布式文件系统的替代方法可以修改您的任务，以便将计算结果发送到某个存储空间，例如数据库，ftp服务器等。

As an alternative to distributed file system, you can modify your task so that it sends the result of the computation to a certain storage, e.g. a database, a ftp server, etc.

这篇关于在Mesos上的Docker：Volume放在哪个节点上？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！