监控Mathematica中并行计算的进度 [英] Monitoring progress of a parallel computation in Mathematica

问题描述

我正在构建一个大型的ParallelTable，并希望对计算的运行情况保持一定的了解.对于非并行表，以下代码可以很好地完成工作:

counter = 1;
Timing[
 Monitor[
  Table[
   counter++
  , {n, 10^6}];
 , ProgressIndicator[counter, {0, 10^6}]
 ]
]

，结果为{0.943512, Null}.但是，对于并行情况，必须在内核之间共享counter:

counter = 1;
SetSharedVariable[counter];
Timing[
 Monitor[
  ParallelTable[
   counter++
  , {n, 10^4}];
 , ProgressIndicator[counter, {0, 10^4}]
 ]
]

，结果为{6.33388, Null}.由于counter的值需要在每次更新时在内核之间来回传递，因此对性能的影响已不至于严重.关于如何进行计算的任何想法?也许让每个内核都有自己的counter值并定期对其求和?也许有一些方法可以确定表中的哪些元素已经被移植到内核中了?

解决方案

当您说也许让每个内核都有自己的计数器值并定期对其求和吗?"时，您几乎自己给出了答案.

尝试这样的事情:

counter = 1;
SetSharedVariable[counter];
ParallelEvaluate[last = AbsoluteTime[]; localcounter = 1;]
Timing[Monitor[
  ParallelTable[localcounter++; 
    If[AbsoluteTime[] - last > 1, last = AbsoluteTime[]; 
     counter += localcounter; localcounter = 0;], {n, 10^6}];, 
  ProgressIndicator[counter, {0, 10^6}]]]

请注意，它比第一个单CPU情况花费的时间更长，这仅是因为它实际上在循环中执行了某些操作.

您可以将测试AbsoluteTime []-last> 1更改为更频繁的内容，例如AbsoluteTime []-last> 0.1.

I'm building a large ParallelTable, and would like to maintain some sense of how the computation is going. For a non parallel table the following code does a great job:

counter = 1;
Timing[
 Monitor[
  Table[
   counter++
  , {n, 10^6}];
 , ProgressIndicator[counter, {0, 10^6}]
 ]
]

with the result {0.943512, Null}. For the parallel case, however, it's necessary to make the counter shared between the kernels:

counter = 1;
SetSharedVariable[counter];
Timing[
 Monitor[
  ParallelTable[
   counter++
  , {n, 10^4}];
 , ProgressIndicator[counter, {0, 10^4}]
 ]
]

with the result {6.33388, Null}. Since the value of counter needs to be passed back and forth between the kernels at every update, the performance hit is beyond severe. Any ideas for how to get some sense of how the computation is going? Perhaps letting each kernel have its own value for counter and summing them at intervals? Perhaps some way of determining what elements of the table have already been farmed out to the kernels?

解决方案

You nearly gave the answer yourself, when you said "Perhaps letting each kernel have its own value for counter and summing them at intervals?".

Try something like this:

counter = 1;
SetSharedVariable[counter];
ParallelEvaluate[last = AbsoluteTime[]; localcounter = 1;]
Timing[Monitor[
  ParallelTable[localcounter++; 
    If[AbsoluteTime[] - last > 1, last = AbsoluteTime[]; 
     counter += localcounter; localcounter = 0;], {n, 10^6}];, 
  ProgressIndicator[counter, {0, 10^6}]]]

Note that it takes longer than your first single-CPU case only because it actually does something in the loop.

You can change the test AbsoluteTime[] - last > 1 to something more frequent like AbsoluteTime[] - last > 0.1.

这篇关于监控Mathematica中并行计算的进度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！