Java函数在MATLAB中的非线性表现 [英] Non-linear performance of Java function in parallel MATLAB

问题描述

最近，我在 MATLAB 程序中实现了并行处理，对于 在MATLAB中慢速xlsread 。然而，实现并行性又出现了另一个问题 - 随着规模的扩大非线性地增加了处理时间。

罪魁祸首似乎是 java.util.concurrent.LinkedBlockingQueue 方法，可以从配置文件的附加图像和相应的压缩图中看到。

问题：如何消除这种非线性，因为我的工作涉及在单次运行中处理超过1000张纸张 - 这需要很长时间？

$ b $注意：程序的并行部分包括读取所有的 .xls 文件并将它们存储在矩阵中，之后我开始我的程序的其余部分。 dlmwrite 用于程序结束，并且对其时间进行优化并不是真正需要的，虽然也可以提示。

---

---

---

元凶：

正在平行的代码：

  parfor i = 1 ：运行
 sin =片; 
 sno = num2str（i）; 
 sna = strcat（sin，sno）; 
 
 data（i，：，:) = xlsread（'Processes.xls'，sna，''，'basic'）; 
 end 
  

解决方案

成为一个问题（事实上可能会变慢），除非你把所有东西放在SSD上。如果您总是阅读相同的文件，但这并不是很大，那么您可能需要在循环之前尝试阅读它，并且只是并行处理数据。

Recently, I implemented parallelisation in my MATLAB program, much to the suggestions offered in Slow xlsread in MATLAB. However, implementing the parallelism has cropped up another problem - non-linearly increasing processing time with increasing scale.

The culprit seems to be the java.util.concurrent.LinkedBlockingQueue method as can be seen from the attached images of profiler and the corresponding condensed graphs.

Problem: How do I remove this non-linearity as my work involves processing more than 1000 sheets in single run - which would take an insanely long time?

Note: The parallelised part of the program involves just reading all the .xls files and storing them in matrices, after which I start the remainder of my program. dlmwrite is used towards the end of the program and optimization on its time is not really required, although could also be suggested.

---

---

---

Culprit:

Code being parallelised:

parfor i = 1:runs
    sin = 'Sheet';
    sno = num2str(i);
    sna = strcat(sin, sno);

    data(i, :, :) = xlsread('Processes.xls', sna, '' , 'basic');
end

解决方案

Doing parallel IO operation is likely to be a problem (could be slower in fact) unless maybe if you keep everything on an SSD. If you are always reading the same file and it's not enormous, you may want to try reading it prior to your loop and just doing your data manipulation in parallel.

这篇关于Java函数在MATLAB中的非线性表现的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！