并行执行多个作业时指定SLURM资源 [英] Specifying SLURM Resources When Executing Multiple Jobs in Parallel

问题描述

根据此处的答案SLURM中的--ntasks或-n任务有什么作用?一个人可以通过sbatch的 ntasks 参数并随后执行 srun 来并行运行多个作业.问一个后续问题-像这样并行运行作业时，如何指定所需的内存量?

According to the answers here What does the --ntasks or -n tasks does in SLURM? one can run multiple jobs in parallel via ntasks parameter for sbatch followed by srun. To ask a follow up question - how would one specify the amount of memory needed when running jobs in parallel like so?

如果说有3个作业并行运行，每个作业需要8G内存，则可以在 sbatch 中指定24G内存(即所有作业的内存总和)，或者不在中指定内存参数> sbatch ，而是为每个 srun 指定8G的内存?

If say 3 jobs are running in parallel each needing 8G of memory, would one specify 24G of memory in sbatch(i.e. the sum of memory from all jobs) or not give memory parameters in sbatch but instead specify 8G of memory for each srun?

推荐答案

您需要在通过 sbatch 提交的脚本中指定内存要求，否则最终将获得默认的内存分配，即可能不符合您的需求.如果随后在 srun 调用中指定8GB内存，那么如果默认内存低于该内存，则可能无法启动任何作业，或者如果并行运行一个或两个作业，则最终可能无法启动默认内存在16到24GB之间.

You need to specify the memory requirement in the script submitted with sbatch, otherwise you will end up with the default memory allocation, which might not correspond to your needs. If you then specify the 8GB memory in the srun call, you might end up with no jobs being able to start if the default memory is lower than that, or having only one or two jobs running in parallel if the default memory is between 16 and 24GB.

您可以请求-mem = 24GB ，但与指定-mem-per-cpu = 8G 相比，灵活性要差.

You can request --mem=24GB, but that offer less flexibility than specifying --mem-per-cpu=8G.

这篇关于并行执行多个作业时指定SLURM资源的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！