Apache Mesos是否可以识别GPU内核? [英] Does Apache Mesos recognize GPU cores?

问题描述

Twitter公开负责人的此演讲的幻灯片25发言人说，Mesos甚至可以跟踪和管理GPU(我假设他指的是GPGPU)资源.但我在其他任何地方都找不到有关此的任何信息.有人可以帮忙吗?除了Mesos，还有其他支持GPGPU的集群管理器吗?

In slide 25 of this talk by Twitter's Head of Open Source office, the presenter says that Mesos allows one to track and manage even GPU (I assume he meant GPGPU) resources. But I cant find any information on this anywhere else. Can someone please help? Besides Mesos, are there other cluster managers that support GPGPU?

推荐答案

Mesos尚未为(GP)GPU提供直接支持，但支持自定义资源类型.如果在启动mesos-slave时指定--resources="gpu(*):8"，则这将成为框架资源的一部分，框架可以启动声称使用这些资源的任务.一旦某个任务使用了某些gpu资源，将仅再次提供其余资源，直到该任务完成并且gpu资源再次变为可用.这样，Mesos资源分配器实际上可以调度您声明的gpu资源，并确保仅将声明的数量提供/分配给框架.

Mesos does not yet provide direct support for (GP)GPUs, but does support custom resource types. If you specify --resources="gpu(*):8" when starting the mesos-slave, then this will become part of the resource offer to frameworks, which can launch tasks that claim to use these resources. Once some of the gpu resources are in use by a task, only the remaining resources will be offered again, until that task completes and the gpu resources become available again. In this way, the Mesos resource allocator can actually schedule the gpu resources you declared, and ensure that only the amount declared are offered/allocated to frameworks.

Mesos尚不支持gpu隔离，但是使用可插拔隔离器模块"，您可以构建自己的gpu隔离器以强制执行gpu资源限制.

Mesos does not yet have support for gpu isolation, but with "pluggable isolator modules", you could build your own gpu isolator to enforce gpu resource limits.

或者，如果您不想分配单个的gpu资源，而只想声明某些节点具有gpu，而另一些则没有，则可以使用--attributes="hasGpu:true"或类似的方法来区分执行/执行的节点没有GPU.此信息也通过资源提供传递给框架，但是这些属性不能被正在运行的任务消耗"，因此将始终为该节点提供这些属性.

Alternately, if you don't want to allocate individual gpu resources, but only want to declare some nodes as having gpus while others do not, you can just use --attributes="hasGpu:true" or something similar to differentiate the nodes that do/do not have gpus. This information is also passed onto the frameworks in resource offers, but these attributes cannot be "consumed" by a running task, so they will always be offered for that node.

有关更多信息，请参见 https://mesos.apache.org/documentation/attributes-资源/

For more information, see https://mesos.apache.org/documentation/attributes-resources/

这篇关于Apache Mesos是否可以识别GPU内核?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！