我们是否可以创建同时具有多个CPU机器和多个GPU机器的Dask集群？ [英] Can we create a Dask cluster having multiple CPU machines as well as multiple GPU machines both.?

问题描述

我们可以创建一个包含一些CPU和一些GPU机器的dask-cluster吗？如果是，如何控制某个任务必须仅在CPU机器上运行，或者某些其他类型的任务应仅在GPU机器上运行，并且如果未指定，则应选择任意空闲的机器。？

Can we create a dask-cluster with some CPU and some GPU machines together. If yes, how to control a certain task must run only on CPU machine, or some other type of task should run only on GPU machine, and if not specified, it should pick whichever machine is free.?

dask是否支持此类群集。什么是控制任务在特定CPU / GPU机器上运行的命令？？

does dask support such type of cluster.? what is the command that controls the task to run on a specific CPU/GPU machine.?

推荐答案

您可以指定Dask工作者具有某些抽象资源

You can specify that a Dask worker has certain abstract resources

dask-worker scheduler:8786 --resources "GPU=2"
dask-worker scheduler:8786 --resources "GPU=2"
dask-worker scheduler:8786 --resources "MEMORY=100e9"

，并且任务在执行期间会消耗这些资源。

and that a task consumes those resources during execution.

processed = [client.submit(process, d, resources={'GPU': 1}) for d in data]
final = client.submit(aggregate, processed, resources={'MEMORY': 70e9})

您可以使用它来为带有GPU的计算机建模。请注意，这些术语GPU和MEMORY只是抽象术语。他们可能同样容易成为FOO和BAR。

You can use this to model machines with GPUs. Note that these terms GPU and MEMORY are just abstract terms. They could just as easily been FOO and BAR.

请参见有关工作人员资源的文档以获取更多信息。

See documentation on worker resources for more information.

这篇关于我们是否可以创建同时具有多个CPU机器和多个GPU机器的Dask集群？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！