有没有办法跟踪 mclapply 的进度? [英] Is there way to track progress on a mclapply?

问题描述

我喜欢 plyr's llply 中的设置 .progress = 'text'.然而，这让我非常焦虑，不知道 mclapply(来自包 multicore)有多远，因为列表项被发送到各个核心，然后在最后进行整理.

I love the setting .progress = 'text' in plyr's llply. However, it causes my much anxiety to not know how far along an mclapply (from package multicore) is since list items are sent to various cores and then collated at the end.

我一直在输出诸如 *currently in sim_id # ....* 之类的消息，但这不是很有帮助，因为它没有告诉我列表项完成的百分比(虽然知道我的脚本没有卡住并继续前进是有帮助的).

I've been outputting messages like *currently in sim_id # ....* but that's not very helpful because it doesn't give me an indicator of what percentage of list items are complete (although it is helpful to know that that my script isn't stuck and moving along).

有人可以提出其他想法，让我查看我的 .Rout 文件并获得进步感吗?我想过添加一个手动计数器，但不知道如何实现它，因为 mclapply 必须完成所有列表项的处理才能给出任何反馈.

Can someone suggest other ideas that would allow me to look at my .Rout file and get a sense of progress? I've thought about adding a manual counter but can't see how I would implement that since mclapply must finish processing all list items before it can give out any feedback.

推荐答案

由于 mclapply 产生多个进程这一事实，人们可能想要使用 fifos、管道甚至套接字.现在考虑以下示例:

Due to the fact that mclapply spawns multiple processes, one might want to use fifos, pipes, or even sockets. Now consider the following example:

library(multicore)

finalResult <- local({
    f <- fifo(tempfile(), open="w+b", blocking=T)
    if (inherits(fork(), "masterProcess")) {
        # Child
        progress <- 0.0
        while (progress < 1 && !isIncomplete(f)) {
            msg <- readBin(f, "double")
            progress <- progress + as.numeric(msg)
            cat(sprintf("Progress: %.2f%%\n", progress * 100))
        } 
        exit()
    }
    numJobs <- 100
    result <- mclapply(1:numJobs, function(...) {
        # Dome something fancy here
        # ...
        # Send some progress update
        writeBin(1/numJobs, f)
        # Some arbitrary result
        sample(1000, 1)
    })
    close(f)
    result
})

cat("Done\n")

这里使用一个临时文件作为fifo，主进程fork一个子进程，它唯一的职责就是报告当前的进度.主进程继续调用 mclapply，其中要评估的表达式(更准确地说，表达式块)通过 writeBin 将部分进度信息写入 fifo 缓冲区.

Here, a temporary file is used as fifo, and the main process forks a child whose only duty is to report the current progress. The main process continues by calling mclapply where the expression (more precisely, the expression block) that is to be evaluated writes partial progress information to the fifo buffer by means of writeBin.

由于这只是一个简单的示例，您可能需要根据自己的需要调整整个输出内容.HTH！

As this is only a simple example, you'll probably have to adapt the whole output stuff to your needs. HTH!

这篇关于有没有办法跟踪 mclapply 的进度?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！