如果我尝试使用比我更多的内核会怎样? [英] What happens if I try to use more cores than I have?

问题描述

在我的sparkconf中，我可以设置要使用的内核数，我的笔记本电脑上有4个物理逻辑，8个逻辑，如果我指定机器上不可能的数字(例如100个内核)，spark会做什么?

In my sparkconf, i can set the number of cores to use, i have 4 physical, 8 logical on my laptop, what does spark do if I specify a number that was not possible on the machine, like say 100 cores?

推荐答案

核数不是描述物理核，而是描述了多个正在运行的线程.这意味着，如果数量大于可用内核的数量，则不会发生任何真正奇怪的事情.

Number of cores doesn't describe physical cores but a number of running threads. It means that nothing really strange happens if the number is higher than a number of available cores.

根据您的设置，它实际上可能是首选配置，其值通常是推荐的可用内核数量的两倍左右.显然，如果数量过多，您的应用程序将在线程之间切换上花费的时间比实际处理花费的时间更多.

Depending on your setup it can be actually a preferred configuration with value around twice a number of available cores being a commonly recommended setting. Obviously if number is to high your application will spend more time on switching between threads than actual processing.

这篇关于如果我尝试使用比我更多的内核会怎样?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！