识别 R 函数和脚本的依赖关系 [英] Identifying dependencies of R functions and scripts
问题描述
我正在筛选一个包和使用该包的脚本,并希望识别外部依赖项.目标是修改脚本以指定library(pkgName)
,并修改包中的函数以使用require(pkgName)
,这样以后这些依赖关系会更加明显.
I am sifting through a package and scripts that utilize the package, and would like to identify external dependencies. The goal is to modify scripts to specify library(pkgName)
and to modify functions in the package to use require(pkgName)
, so that these dependencies will be more obvious later.
我正在修改代码以考虑每个外部依赖包.作为一个例子,虽然它绝不是确定的,但我现在发现很难识别依赖于 data.table
的代码.我可以用 Matrix
、ggplot2
、bigmemory
、plyr
替换 data.table
,或许多其他包,因此请随意回答基于其他包的示例.
I am revising the code to account for each externally dependent package. As an example, though it is by no means definitive, I am now finding it difficult to identify code that depends on data.table
. I could replace data.table
with Matrix
, ggplot2
, bigmemory
, plyr
, or many other packages, so feel free to answer with examples based on other packages.
这个搜索并不容易.到目前为止我尝试过的方法包括:
This search isn't particularly easy. The approaches I have tried so far include:
- 在代码中搜索
library
和require
语句 - 搜索提及
data.table
(例如library(data.table)
) - 尝试运行
codetools::checkUsage
以确定可能存在问题的位置.对于脚本,我的程序将脚本插入本地函数并将checkUsage
应用于该函数.否则,我使用checkUsagePackage
作为包. - 寻找对
data.table
有点独特的语句,例如:=
. - 寻找可以通过匈牙利符号(例如
DT
标识对象的类的位置
- Search the code for
library
andrequire
statements - Search for mentions of
data.table
(e.g.library(data.table)
) - Try running
codetools::checkUsage
to determine where there may be some issues. For the scripts, my program inserts the script into a local function and appliescheckUsage
to that function. Otherwise, I usecheckUsagePackage
for the package. - Look for statements that are somewhat unique to
data.table
, such as:=
. - Look for where objects' classes may be identified via Hungarian notation, such as
DT
我搜索的本质是找到:
- 加载
data.table
, - 对象名称表明它们是
data.table
对象, - 似乎是
data.table
特定的方法
- loading of
data.table
, - objects with names that indicate they are
data.table
objects, - methods that appear to be
data.table
-specific
其中唯一简单的部分似乎是找到包的加载位置.不幸的是,并非所有函数都可以显式加载或需要外部包 - 这些可能假定它已经加载.这是一个不好的做法,我正在尝试解决它.但是,搜索对象和方法似乎具有挑战性.
The only easy part of this seems to be finding where the package is loaded. Unfortunately, not all functions may explicitly load or require the external package - these may assume it has already been loaded. This is a bad practice, and I am trying to fix it. However, searching for objects and methods seems to be challenging.
这(data.table
)只是一个包,一个似乎有限且有些独特的用法.假设我想寻找 ggplot 函数的用途,其中选项更广泛,并且语法文本没有那么特殊(即 +
的频繁使用不是特殊的,而 :=
似乎是).
This (data.table
) is just one package, and one with what seems to be limited and somewhat unique usage. Suppose I wanted to look for uses of ggplot functions, where the options are more extensive, and the text of the syntax is not as idiosyncratic (i.e. frequent usage of +
is not idiosyncratic, while :=
seems to be).
我认为静态分析不会给出完美的答案,例如可以将参数传递给函数,该函数指定要加载的包.尽管如此:是否有任何核心工具或软件包可以通过静态或动态分析来改进这种蛮力方法?
I don't think that static analysis will give a perfect answer, e.g. one could pass an argument to a function, which specifies a package to be loaded. Nonetheless: are there any core tools or packages that can improve on this brute force approach, either via static or dynamic analysis?
对于它的价值,tools::pkgDepends
仅解决包级别的依赖关系,而不是函数或脚本级别,这是我正在工作的级别.
For what it's worth, tools::pkgDepends
only addresses dependencies at the package level, not the function or script level, which is the level I'm working at.
更新 1:应该工作的动态分析工具的一个示例是报告在代码执行期间加载了哪些包.不过,我不知道 R 中是否存在这样的功能 - 这就像 Rprof
报告 search()
的输出而不是代码堆栈.
Update 1: An example of a dynamic analysis tool that should work is one that reports which packages are loaded during code execution. I don't know if such a capability exists in R, though - it would be like Rprof
reporting the output of search()
instead of the code stack.
推荐答案
首先,感谢@mathematical.coffee 让我走上了使用 Mark Bravington 的 mvbutils
包的道路.foodweb
功能非常令人满意.
First, thanks to @mathematical.coffee to putting me on the path of using Mark Bravington's mvbutils
package. The foodweb
function is more than satisfactory.
回顾一下,我想知道如何检查一个包,例如 myPackage
与另一个包,例如 externalPackage
,以及检查脚本是否与 externalPackage代码>.我将演示如何做每一个.在这种情况下,外部包是
data.table
.
To recap, I wanted to know about about checking one package, say myPackage
versus another, say externalPackage
, and about checking scripts against the externalPackage
. I'll demonstrate how to do each. In this case, the external package is data.table
.
1:对于 myPackage
与 data.table
,以下命令就足够了:
1: For myPackage
versus data.table
, the following commands suffice:
library(mvbutils)
library(myPackage)
library(data.table)
ixWhere <- match(c("myPackage","data.table"), search())
foodweb(where = ixWhere, prune = ls("package:data.table"), descendents = FALSE)
这会生成一个出色的图表,显示哪些函数依赖于 data.table
中的函数.虽然该图包含 data.table
中的依赖关系,但它并不过分繁琐:我可以很容易地看到我的哪些函数依赖于 data.table
,以及它们使用了哪些函数,例如如 as.data.table
、data.table
、:=
、key
等.在这一点上,可以说包依赖问题已经解决,但是 foodweb
提供的东西太多了,让我们来看看.最酷的部分是依赖矩阵.
This produces an excellent graph showing which functions depend on functions in data.table
. Although the graph includes dependencies within data.table
, it's not overly burdensome: I can easily see which of my functions depend on data.table
, and which functions they use, such as as.data.table
, data.table
, :=
, key
, and so on. At this point, one could say the package dependency problem is solved, but foodweb
offers so much more, so let's look at that. The cool part is the dependency matrix.
depMat <- foodweb(where = ixWhere, prune = ls("package:data.table"), descendents = FALSE, plotting = FALSE)
ix_sel <- grep("^myPackage.",rownames(depMat))
depMat <- depMat[ix_sel,]
depMat <- depMat[,-ix_sel]
ix_drop <- which(colSums(depMat) == 0)
depMat <- depMat[,-ix_drop]
ix_drop <- which(rowSums(depMat) == 0)
depMat <- depMat[-ix_drop,]
这很酷:它现在显示了我的包中函数的依赖关系,我在其中使用了详细的名称,例如myPackage.cleanData
,关于函数不是在我的包中,即 data.table
中的函数,它消除了没有依赖关系的行和列.这很简洁,让我可以快速调查依赖关系,并且通过处理 rownames(depMat)
,我也可以很容易地找到我的函数的补充集.
This is cool: it now shows dependencies of functions in my package, where I'm using verbose names, e.g. myPackage.cleanData
, on functions not
in my package, namely functions in data.table
, and it eliminates rows and columns where there are no dependencies. This is concise, lets me survey dependencies quickly, and I can find the complementary set for my functions quite easily, too, by processing rownames(depMat)
.
注意:plotting = FALSE
似乎不会阻止创建绘图设备,至少在第一次调用 foodweb
时是这样.这很烦人,但并不可怕.也许我做错了什么.
NB: plotting = FALSE
doesn't seem to prevent a plotting device from being created, at least the first time that foodweb
is called in a sequence of calls. That is annoying, but not terrible. Maybe I'm doing something wrong.
2:对于脚本与 data.table
,这会变得更有趣一些.对于每个脚本,我需要创建一个临时函数,然后检查依赖项.我在下面有一个小功能,正是这样做的.
2: For scripts versus data.table
, this gets a little more interesting. For each script, I need to create a temporary function, and then check for dependencies. I have a little function below that does precisely that.
listFiles <- dir(pattern = "myScript*.r")
checkScriptDependencies <- function(fname){
require(mvbutils)
rawCode <- readLines(fname)
toParse <- paste("localFunc <- function(){", paste(rawCode, sep = "
", collapse = "
"), "}", sep = "
", collapse = "")
newFunc <- eval(parse(text = toParse))
ix <- match("data.table",search())
vecPrune <- c("localFunc", ls("package:data.table"))
tmpRes <- foodweb(where = c(environment(),ix), prune = vecPrune, plotting = FALSE)
tmpMat <- tmpRes$funmat
tmpVec <- tmpMat["localFunc",]
return(tmpVec)
}
listDeps <- list()
for(selFile in listFiles){
listDeps[[selFile]] <- checkScriptDependencies(selFile)
}
现在,我只需要看一下 listDeps
,我就有了与上面的 depMat 相同的奇妙的小见解.我从我编写的其他代码修改了 checkScriptDependencies
,这些代码发送要由 codetools::checkUsage
分析的脚本;有一个像这样的小函数来分析独立代码是很好的.感谢 @Spacedman 和 @Tommy 了解使用 environment()
改进了对 foodweb
的调用.
Now, I just need to look at listDeps
, and I have the same kind of wonderful little insights that I have from the depMat above. I modified checkScriptDependencies
from other code that I wrote that sends scripts to be analyzed by codetools::checkUsage
; it's good to have a little function like this around for analyzing standalone code. Kudos to @Spacedman and @Tommy for insights that improved the call to foodweb
, using environment()
.
(真正的匈牙利人会注意到我与名称和类型的顺序不一致 - 太糟糕了.:) 这有一个更长的原因,但这并不是我正在使用的代码.)
(True hungaRians will notice that I was inconsistent with the order of name and type - tooBad. :) There's a longer reason for this, but this isn't precisely the code I'm using, anyway.)
虽然我没有发布 foodweb
为我的代码生成的图表的图片,但您可以在 http://web.archive.org/web/20120413190726/http://www.sigmafield.org/2010/09/21/r-function-of-the-day-foodweb.在我的例子中,它的输出肯定会捕获 data.table 对 :=
和 J
的使用,以及标准命名函数,例如 key
和 <代码>as.data.table.它似乎避免了我的文本搜索,并且在几个方面都有改进(例如,查找我忽略的功能).
Although I've not posted pictures of the graphs produced by foodweb
for my code, you can see some nice examples at http://web.archive.org/web/20120413190726/http://www.sigmafield.org/2010/09/21/r-function-of-the-day-foodweb. In my case, its output definitely captures data.table's usage of :=
and J
, along with the standard named functions, like key
and as.data.table
. It seems to obviate my text searches and is an improvement in several ways (e.g. finding functions that I'd overlooked).
总而言之,foodweb
是一个出色的工具,我鼓励其他人探索 mvbutils
包和 Mark Bravington 的一些其他不错的包,例如 调试
.如果您确实安装了 mvbutils
,如果您认为只有自己在为管理不断发展的 R 代码而苦恼,请查看 ?changed.funs
.:)
All in all, foodweb
is an excellent tool, and I encourage others to explore the mvbutils
package and some of Mark Bravington's other nice packages, such as debug
. If you do install mvbutils
, just check out ?changed.funs
if you think that only you struggle with managing evolving R code. :)
这篇关于识别 R 函数和脚本的依赖关系的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!