R并行makeCluster()在Mac上无限期挂起 [英] R parallel makeCluster() hangs infinitely on Mac

问题描述

当我尝试在Mac上的R中使用parallel软件包时遇到了一些问题.

I came across some problem when I was trying to use parallel package in R on my Mac.

这是并行程序包正常工作的方式.

Here is how the parallel package works normally.

cl = makeCluster(2) # Using 2-core parallel as an example
# Your parallel code
stopCluster(cl)

当我运行这段代码时，cl = makeCluster(2) 无限挂起.我试图解决它，但是失败了.我还提到了其他一些帖子.几个潜在的原因包括内存不足，安装错误等.当我重新启动会话，重新安装R时，它们似乎不是问题所在，但问题仍然存在.

When I ran this code, the cl = makeCluster(2) hangs infinitely. I was trying to solve it but failed. I also referred to some other posts. Several potential reasons includes not enough memory, installation error, etc. They do not seems to be the problem here, as I restarted sessions, reinstalled R, but the problem remained.

我猜问题出在R尝试连接内核时的权限问题.这是我发现的.我使用了future包来查看连接到内核的具体过程.随附的是代码及其返回.

I guess the problem is about the permission when R tried to connect to cores. Here is what I found out. I used future package to see the specific process of connecting to cores. Attached are the code and its return.

cl <- future::makeClusterPSOCK(2, verbose = TRUE)

工人:[n = 2]本地主机"，本地主机" 基本端口:11303 创建2个节点中的1个... -设置节点 在'localhost'上启动工作程序#1:'/Library/Frameworks/R.framework/Resources/bin/Rscript'-默认程序包=数据集，utils，grDevices，图形，统计信息，方法-e'parallel :::.slaveRSOCK ()'MASTER = localhost PORT = 11303 OUT =/dev/null TIMEOUT = 2592000 XDR = TRUE
等待本地主机"上的工作人员#1重新连接

Workers: [n = 2] ‘localhost’, ‘localhost’ Base port: 11303 Creating node 1 of 2 ... - setting up node Starting worker #1 on ‘localhost’: '/Library/Frameworks/R.framework/Resources/bin/Rscript' --default packages=datasets,utils,grDevices,graphics,stats,methods -e 'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11303 OUT=/dev/null TIMEOUT=2592000 XDR=TRUE
Waiting for worker #1 on ‘localhost’ to connect back

问题是本地主机永不回连接...

The problem is the localhost never connects back ...

以下是我的会话信息.我希望这有帮助.

The following my the session info. I hope this helps.

R版本3.5.1(2018-07-02).
平台:x86_64-apple-darwin15.6.0(64位).
运行于:macOS High Sierra 10.13.6.

R version 3.5.1 (2018-07-02).
Platform: x86_64-apple-darwin15.6.0 (64-bit).
Running under: macOS High Sierra 10.13.6.

Matrix产品:默认值.
BLAS:/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib.
LAPACK:/Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

Matrix products: default.
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib.
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

语言环境:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

附加的基本软件包:
[1]统计图形grDevices utils数据集方法基础

attached base packages:
[1] stats graphics grDevices utils datasets methods base

通过名称空间(未附加)加载:
[1]编译器_3.5.1并行_3.5.1工具_3.5.1侦听_0_0.7.0 codetools_0.2-15摘要_0.6.16
[7] globals_0.12.2 future_1.9.0

loaded via a namespace (and not attached):
[1] compiler_3.5.1 parallel_3.5.1 tools_3.5.1 listenv_0.7.0 codetools_0.2-15 digest_0.6.16
[7] globals_0.12.2 future_1.9.0

有趣的是，相同的代码可以在我的旧Mac机器上运行(相同的OS，但硬件更旧).我不知道这里发生了什么.任何帮助表示赞赏！谢谢！

It's interesting that the same code works on my old Mac machine (same OS but the hardware is older). I have no idea what is happening here. Any help is appreciated! Thanks!

推荐答案

许多潜在的原因包括内存不足，安装错误等.当我重新启动会话，重新安装R时，它们似乎不是问题所在.

Several potential reasons includes not enough memory, installation error, etc. They do not seems to be the problem here, as I restarted sessions, reinstalled R, but the problem remained.

正确的，这里不应该涉及那些类型的问题.您显示的调用使用R的基本内置功能(大部分来自并行"包)，并且涉及的内存使用很少.

Correct, those type of problems should not be involved here. The calls you've shown use basic built-in functionalities of R (mostly from the 'parallel' package) and there's very little memory usage involved.

我猜问题出在R尝试连接内核时的权限问题. [...]

I guess the problem is about the permission when R tried to connect to cores. [...]

parallel:makeCluster(2)和future::makeClusterPSOCK(2)都启动工作程序(使用parallel:::.slaveRSOCK())，这些工作程序是在后台运行的独立R会话.主会话和这些工作程序通过套接字进行通信.因此，是的，可能是您的防火墙出现问题，导致R无法打开这些端口. (我不知道要解决此问题的macOS足够多)

Both parallel:makeCluster(2) and future::makeClusterPSOCK(2) launches workers (using the parallel:::.slaveRSOCK()) that are independent R sessions that run in the background. The master session and these workers communicate via sockets. So, yes, it could be that you have firewall issues preventing R from opening those ports. (I don't know enough macOS to troubleshoot that)

通过设置outfile = NULL，您还将获得有关工人端发生的情况的信息.这是它工作时的样子:

By setting outfile = NULL, you will also get information on what happens on the workers' end. Here is what it should look like when it works:

> cl <- future::makeClusterPSOCK(1, outfile = NULL, verbose = TRUE)
Workers: [n = 1] ‘localhost’
Base port: 11306
Creating node 1 of 1 ...
- setting up node
Starting worker #1 on ‘localhost’: '/usr/lib/R/bin/Rscript' --default-packages=datasets,utils,grDevices,graphics,stats,methods -e 'parallel:::.slaveRSOCK()' MASTER=localhost PORT=11306 OUT= TIMEOUT=2592000 XDR=TRUE
Waiting for worker #1 on ‘localhost’ to connect back
starting worker pid=7608 on localhost:11306 at 14:46:57.827
Connection with worker #1 on ‘localhost’ established
- assigning connection UUID
- collecting session information
Creating node 1 of 1 ... done

PS.您只需要一名工人即可解决此问题.

PS. You only need one worker to troubleshoot this.

这篇关于R并行makeCluster()在Mac上无限期挂起的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！