Furrr的“黑魔法"是什么? [英] What is furrr's "black magic?"

问题描述

我使用R包 furrr 满足大多数并行化需求，并且从我的全局环境向集群导出内容基本上没有问题.今天我做了，我不知道为什么.软件包文档似乎描述了将全局变量作为黑魔法"发送到群集的过程.什么是黑魔法?

I use the R package furrr for most of my parallelization needs, and basically never have issues with exporting things from my global environment to the cluster. Today I did and I have no idea why. The package documentation seems to describe the process by which global variables are sent to the clusters as "black magic." What is the black magic?

furrr :: future_options 文档说:

全局变量和包默认情况下，future程序包将执行魔术操作以查找furrr调用所需的全局变量和程序包，并将其导出到每个工作程序.但是，它并不总是完美的，可以使用globals和packages参数对其进行完善.

Global variables and packages By default, the future package will perform black magic to look up the global variables and packages that your furrr call requires, and it will export these to each worker. However, it is not always perfect, and can be refined with the globals and packages arguments.

作为第二个问题:是否有一种优雅的方式告诉它进行黑魔法，但也可以导出它错过的东西?还是选择a)都是黑魔法，或者b)硬编码 .options 参数中的所有内容?

As a secondary question: is there an elegant way to tell it to do its black magic, but also to export something it missed? Or, are the choices a) all black magic, or b) hard code everything in the .options argument?

推荐答案

这不能完全回答问题，但我认为它为您指明了正确的方向.来自全球"此简介小插图的部分:

This doesn't totally answer the question, but I think it points you in the right direction. From the "Globals" section of this intro vignette:

它是在 globals 软件包的帮助下完成的，该软件包使用静态代码检查以识别全局变量.如果识别出全局变量，则将其捕获并提供给评估过程.

It does this with help of the globals package, which uses static-code inspection to identify global variables. If a global variable is identified, it is captured and made available to the evaluating process.

还有此常见问题"与解决方案"小插图(上面@michael有用的链接)，讨论了一些常见的陷阱"全局代码包中的静态代码eval导致的结果.

There's also this "Common Issues with Solutions" vignette (which @michael helpfully linked above) that discusses some common "gotcha" things that result from the static code eval in the globals package.

我在这里找到了自己的出路，因为我的 future_map()代码无法找到我在 glue()调用中引用的变量.那个小插图准确地解释了为什么会发生这种情况.

I found my way here because my future_map() code was failing to find the variables I referenced inside a glue() call. That vignette explained exactly why this happens.

关于为什么您的代码有时有效而有时无效的原因，很难说.但是正如您所看到的，引擎盖下的复杂性足够大，如果一些看似无关的更改破坏了某些东西，我并不感到惊讶.(对我而言，此更改是清理代码并使用 glue 而不是 paste :shrug:)

As to why your code was sometimes working and sometimes not, hard to say. But as you can see, there is sufficient complexity going on under the hood that I'm not surprised if some seemingly unrelated change broke something. (For me this change was cleaning up my code and using glue instead of paste :shrug:)

这篇关于Furrr的“黑魔法"是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！