R limited中的并行处理 [英] Parallel processing in R limited

问题描述

我正在运行带有并行和doParallel软件包的ubuntu 12.04和R 2.15.1.当我并行运行任何东西时，我仅限于100％的内核，而我应该拥有800％的内核，因为我使用8​​个内核运行它.系统监视器上显示的是每个子进程仅获得12％.

I'm running ubuntu 12.04 and R 2.15.1 with the parallel and doParallel packages. when I run anything in parallel I'm limited to 100% of a core, when I should have up to 800%, since I am running it with 8 cores. What shows up on the system monitor is that each child process is getting only 12%.

这是在限制我的执行速度吗?

What is going on that is limiting my execution speed?

推荐答案

问题可能是R进程仅限于一个核心(子进程继承了该核心).

The problem may be that the R process is restricted to one core (and the subprocesses inherit that).

尝试一下:

> system(sprintf("taskset -p 0xffffffff %d", Sys.getpid()))
pid 3064's current affinity mask: fff
pid 3064's new affinity mask: fff

现在，如果在您的计算机上，当前的亲和力掩码报告为1，那么这就是问题所在.上面的行应解决此问题(即第二行应报告fff(或类似内容).

Now, if on your machine, the current affinity mask reports a 1, then this was the problem. The line above should solve it (i.e. the second line should report fff (or similar).

Simon Urbanek编写了一个函数mcaffinity，该函数允许对多核进行此控制.据我所知，它仍然在R-devel中.

Simon Urbanek wrote a function mcaffinity that allows this control for multicore. As far as I know, it's still in R-devel.

有关详细信息，请参见此对R-sig-hpc的讨论.

For details, see e.g. this discussion on R-sig-hpc.

更新以及对Xin Guo的回答的补充:

Update, and addition to Xin Guo's answer:

如果同时使用通过openblas进行的隐式并行化和通过并行/snow/multicore进行的显式并行化，则可能要更改openblas使用的线程数，具体取决于您是否位于显式并行部分中. > 这是可能的(对于Linux下的openblas，我不知道其他任何经过优化的BLAS都可以对线程数量提供功能)，请参见

If you use implicit parallelization via openblas and explicit parallelization (via parallel/snow/multicore) together, you may want to change the number of threads that openblas uses depending on whether you are inside an explicitly parallel part or not.
This is possible (with openblas under Linux, I'm not aware of any other of the usual optimized BLAS' that provides a function to the number of threads), see Simon Fuller's blog post for details.

这篇关于R limited中的并行处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！