Mesos 的“主"和“从"节点可以部署在同一台机器上吗? [英] Can Mesos 'master' and 'slave' nodes be deployed on the same machines?

问题描述

Apache Mesos 的主"节点可以与 Mesos 的从"节点位于同一台机器上吗?同样(对于高可用性(HA)部署)，Mesos 'master' 选举中使用的 Apache Zookeeper 节点是否可以与 Mesos 'slave' 节点部署在同一台机器上?

Can Apache Mesos 'master' nodes be co-located on the same machine as Mesos 'slave' nodes? Similarly (for high-availability (HA) deploys), can the Apache Zookeeper nodes used in Mesos 'master' election be deployed on the same machines as Mesos 'slave' nodes?

Mesos 建议使用 3 个masters"进行 HA 部署，Zookeeper 建议使用 5 个节点用于其仲裁选举系统.最好让这些服务与 Mesos 的从属"进程一起运行，而不是将 8 台机器用于有效的非生产性"任务.

Mesos recommends 3 'masters' be used for HA deploys, and Zookeeper recommends 5 nodes be used for its quorum election system. It would be nice to have these services running along side Mesos 'slave' processes instead of committing 8 machines to effectively 'non-productive' tasks.

如果这样的设置可行，那么这样的设置有什么优点/缺点?

If such a setup is feasible, what are the pros/cons of such a setup?

谢谢！

推荐答案

你绝对可以在同一个节点上运行一个 master、slave 和 zk 进程.您甚至可以在同一个节点上运行多个主进程和从进程，前提是您为它们提供了每个唯一的端口，但这仅对测试集群有用.

You can definitely run a master, slave, and zk process all on the same node. You can even run multiple master and slave processes on the same node, provided you give them each unique ports, but that's only useful for a test cluster.

通常我们建议在与您的主节点相同的节点上运行 ZK，但如果您有额外的 ZK，您当然可以在从节点上运行它们，或者您认为合适的混合搭配，只要所有主/从/框架节点可以到达ZK节点，所有从节点都可以到达主节点.

Typically we recommend running ZK on the same nodes as your masters, but if you have extra ZKs, you can certainly run them on slaves, or mix-and-match as you see fit, as long as all master/slave/framework nodes can reach the ZK nodes, and all slaves can reach the masters.

对于较小的集群(<10 个节点)，在每个主节点上运行一个从进程是有意义的，特别是因为备用主节点不会做太多事情.即使是小型集群的活动 master 也只使用少量的 cpu、内存和网络资源.只需确保调整该从属服务器上的 --resources 以考虑主服务器的资源使用情况.

For a smaller cluster (<10 nodes) it could make sense to run a slave process on each master, especially since the standby masters won't be doing much. Even an active master for a small cluster uses only a small amount of cpu, memory, and network resources. Just make sure you adjust the --resources on that slave to account for the master's resource usage.

一旦您的集群变大(尤其是 >100 个节点)，进出主节点的网络流量以及它的 cpu/内存利用率变得足够重要，以至于您不想在与主节点相同的节点上运行 mesos 从节点掌握.即使大规模地将 ZK 与您的 master 共置应该没问题.

Once your cluster grows larger (especially >100 nodes) the network traffic to/from the master as well as its cpu/memory utilization becomes significant enough that you wouldn't want to run a mesos slave on the same node as the master. It should be fine to co-locate ZK with your master even at large scale.

您没有具体询问，但我也会讨论在何处运行您的框架调度程序(例如 Spark、Marathon 或 Chronos).这些可以与任何其他组件位于同一位置，但它们只需要真正能够到达主节点和 zk 节点，因为与从节点的所有通信都通过主节点.一些客户在主节点上运行调度程序，一些客户在边缘节点上运行它们(因此用户无法访问从节点)，而其他客户使用 Marathon 等元框架在从节点上运行其他调度程序作为 Mesos 任务.

You didn't specifically ask, but I'll also discuss where to run your framework schedulers (e.g. Spark, Marathon, or Chronos). These could be co-located with any of the other components, but they only really need to be able to reach the master and zk nodes, since all communication to slaves goes through the master. Some customers run the schedulers on master nodes, some run them on edge nodes (so users don't have access to the slaves), and others use meta-frameworks like Marathon to run other schedulers on slaves as Mesos tasks.

这篇关于Mesos 的“主"和“从"节点可以部署在同一台机器上吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！