为什么R多核仅使用一个核? [英] Why is R multicore only using one core?

问题描述

可能重复:
在R受限中进行并行处理

Possible Duplicate:
Parallel processing in R limited

我已经用R多核编写了一些代码，并且正在24核计算机上运行它.实际上只有12个内核，但是它们是超线程的，因此看起来好像有24个内核.

I've written some code in R multicore, and I'm running it on a 24-core machine. In fact there are only 12 cores, but they are hyperthreaded, so it looks like there are 24.

奇怪的是:所有线程都在同一个单核上运行！因此，它们每个都只使用少量的cpu，而不是每个都运行在单个内核上并消耗所有可用的内核.

Here's what's strange: all the threads run on the same single core! So they each only use a tiny amount of cpu, instead of each running on a single core, and chewing up all available cores.

为简单起见，我只运行4个线程:

For simplicity, I'm just running 4 threads:

mclapply( 1:30, function(size) {
    # time consuming stuff that is cpu bound (think "forecast.ets" et al)
}, mc.cores = 4, mc.preschedule = F )

在运行此进程之前，已经在一个内核上运行了一个R进程，并使用该内核的100％的容量:

Prior to running this, there is already an R process running on one core, using 100% of that core's capacity:

接下来，我启动多核进程"，并且有4个额外的线程争用同一核！:

Next, I launch the "multicore process", and 4 extra threads fight for the same core!:

...因此，当他们每个人都应该能够获得一个内核的100％时，他们每个人都将获得一个内核的12％，或大约1％的可用处理能力.另外，其他R进程现在仅获得50％的内核.

... so, they each get 12% of one core, or about 1% of the available processing power, when they should each be able to get 100% of one core. Also, the other R process now only get 50% of the core.

OS是Ubuntu 12.04 64位.硬件是英特尔. R是2.15.2版的捣蛋"

OS is Ubuntu 12.04 64-bit. Hardware is Intel. R is version 2.15.2 "trick or treat"

有什么想法吗? (我知道我可以使用降雪量，但是我有很多变量，而且我真的不想sfExport所有变量！)

Thoughts? (I know I could just use snowfall, but I have a ton of variables, and I really don't want to have to sfExport all of them!)

哦，我想某个地方有一些全局锁?但是，为什么两个完全独立的R进程之间会有冲突?我可以并行运行两个R进程，每个进程占用100％内核的CPU.

oh, I guess there's some global lock somewhere? But still, why would there be a conflict between two completely separate R processes? I can run two R processes in parallel just fine, with each taking 100% of a core's CPU.

Edit2:多亏了Dirk的指导，我重建了openblas，现在看起来更健康了！:

Thanks to Dirk's pointer, I rebuilt openblas, and it's looking much healthier now!:

推荐答案

一个可能的问题是OpenBLAS软件包的可能的副作用，该软件包设置CPU亲和力，使进程停留在一个内核上.请参阅 R限制中的并行处理以进行讨论，并链接到有关r- sig-hpc列表已修复.

A possible issue is a possible side effect of the OpenBLAS package which sets CPU affinity such that processes stick to one core. See Parallel processing in R limited for a discussion and link to more discussion on the r-sig-hpc list which has a fix.

这篇关于为什么R多核仅使用一个核?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！