如何保存parfor循环的各个迭代 [英] How can I save individual iterations of a parfor loop

问题描述

我正在努力保存每个PARFOR迭代的输出，但是直到现在仍未成功.我已经在网上进行了大量搜索，根据一些newgroup的官方回复，处理PARFOR中保存问题的正确方法(即解决透明度"问题)是通过以下方式使用外部函数(基本示例) :

I'm struggling to save output of each PARFOR iteration, but until now without success. I've searched a lot online, and according to some official newgroup replies, the correct way to deal with saving in a PARFOR (i.e., addressing the "transparency" issue) is to use an external function in the following way (elementary example):

parfor i=1:120
  display(i);
  [LL ll] = eig(rand(1000,1000));
  record(:,i) = diag(ll);
  samplesave('save.mat',record(:,i),i);
end

以及

function samplesave(fname, data,i)
   persistent st;
   store(:,i)=data;
   save(fname);
end

问题在于，将"st"声明为PERSISTENT将使同一工作线程的变量持久" 跨调用，但不会跨PARFOR循环 ，因此在在任何给定的点上，您只有最后一个要保存的工作人员的全部历史记录；下一个PARFOR迭代(来自其他工作程序)将使用(仅)该工作程序的整个历史覆盖此记录，依此类推.

the problem is that declaring "st" as PERSISTENT make the variable "persistent" across calls of the same worker, but not across the PARFOR loop, so that at any given point you have only the whole history of the last worker that managed to save; the next PARFOR iteration (from a different worker) will overwrite this record with the whole history of (only) that worker, and so on.

如何增量保存所有工人的历史记录?

How can I incrementally save history of all the workers?

谢谢

推荐答案

我认为有一个原因，您迫不及待要等到循环终止之后再保存整个矩阵.

I assume there is a reason why you can't wait until after the loop terminate to save the entire matrix.

一个想法是使用这样的方法将每个迭代独立于其他迭代保存:

One idea would be to save each iteration independent of the others using something like this:

baseFile = ('part_%03d.mat');
parfor i=1:120
  display(i);
  [LL ll] = eig(rand(1000,1000));
  singleRecord = diag(ll);
  record(:,i) = singleRecord;
  save(sprintf(baseFile, i), 'singleRecord');
end

然后逐个加载文件，然后将值组合到一个矩阵中.

Then load the files one by one and combine the values into a single matrix.

mat = [];
baseFile = ('part_%03d.mat');
for i = 1:120
  d = load(sprintf(baseFile, i));
  mat(:,i) = d.singleRecord;
end

这篇关于如何保存parfor循环的各个迭代的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！