Java和两个double [] []并行流 [英] Java sum two double[][] with parallel stream
问题描述
假设我有两个矩阵：
double[][] a = new double[2][2]
a[0][0] = 1
a[0][1] = 2
a[1][0] = 3
a[1][1] = 4
double[][] b = new double[2][2]
b[0][0] = 1
b[0][1] = 2
b[1][0] = 3
b[1][1] = 4
传统方式，总结这个矩阵我会做一个嵌套for循环：
in the traditional way, to sum this matrices I would do a nested for loop:
int rows = a.length;
int cols = a[0].length;
double[][] res = new double[rows][cols];
for(int i = 0; i < rows; i++){
for(int j = 0; j < cols; j++){
res[i][j] = a[i][j] + b[i][j];
}
}
我对流API很新，但是我认为这非常适合与 parallelStream 一起使用，所以我的问题是，如果有办法做到这一点并利用并行处理？
I'm fairly new to the stream API but I think this is a great fit to use with parallelStream so my question is if there's a way to do this and take advantage of parallel processing?
编辑：不确定这是不是正确的地方，但我们继续：
使用一些建议我将Stream推送到测试中。设置如下：
经典方法：
not sure if this is the right place but here we go: Using some suggestions I putted the Stream to the test. The set up was like this: Classical approach:
public class ClassicMatrix {
private final double[][] components;
private final int cols;
private final int rows;
public ClassicMatrix(final double[][] components){
this.components = components;
this.rows = components.length;
this.cols = components[0].length;
}
public ClassicMatrix addComponents(final ClassicMatrix a) {
final double[][] res = new double[rows][cols];
for (int i = 0; i < rows; i++) {
for (int j = 0; j < rows; j++) {
res[i][j] = components[i][j] + a.components[i][j];
}
}
return new ClassicMatrix(res);
}
}
使用@dkatzel建议：
Using @dkatzel suggestion:
public class MatrixStream1 {
private final double[][] components;
private final int cols;
private final int rows;
public MatrixStream1(final double[][] components){
this.components = components;
this.rows = components.length;
this.cols = components[0].length;
}
public MatrixStream1 addComponents(final MatrixStream1 a) {
final double[][] res = new double[rows][cols];
IntStream.range(0, rows*cols).parallel().forEach(i -> {
int x = i/rows;
int y = i%rows;
res[x][y] = components[x][y] + a.components[x][y];
});
return new MatrixStream1(res);
}
}
使用@Eugene建议：
Using @Eugene suggestion:
public class MatrixStream2 {
private final double[][] components;
private final int cols;
private final int rows;
public MatrixStream2(final double[][] components) {
this.components = components;
this.rows = components.length;
this.cols = components[0].length;
}
public MatrixStream2 addComponents(final MatrixStream2 a) {
final double[][] res = new double[rows][cols];
IntStream.range(0, rows)
.forEach(i -> Arrays.parallelSetAll(res[i], j -> components[i][j] * a.components[i][j]));
return new MatrixStream2(res);
}
}
和一个测试类，运行3次独立时间一次每个方法（只需替换main（）中的方法名称）：
and a test class, running 3 independent times one for each method (just replacing the method name in main()):
public class MatrixTest {
private final static String path = "/media/manuel/workspace/data/";
public static void main(String[] args) {
final List<Double[]> lst = new ArrayList<>();
for (int i = 100; i < 8000; i = i + 400) {
final Double[] d = testClassic(i);
System.out.println(d[0] + " : " + d[1]);
lst.add(d);
}
IOUtils.saveToFile(path + "classic.csv", lst);
}
public static Double[] testClassic(final int i) {
final ClassicMatrix a = new ClassicMatrix(rand(i));
final ClassicMatrix b = new ClassicMatrix(rand(i));
final long start = System.currentTimeMillis();
final ClassicMatrix mul = a.addComponents(b);
final long now = System.currentTimeMillis();
final double elapsed = (now - start);
return new Double[] { (double) i, elapsed };
}
public static Double[] testStream1(final int i) {
final MatrixStream1 a = new MatrixStream1(rand(i));
final MatrixStream1 b = new MatrixStream1(rand(i));
final long start = System.currentTimeMillis();
final MatrixStream1 mul = a.addComponents(b);
final long now = System.currentTimeMillis();
final double elapsed = (now - start);
return new Double[] { (double) i, elapsed };
}
public static Double[] testStream2(final int i) {
final MatrixStream2 a = new MatrixStream2(rand(i));
final MatrixStream2 b = new MatrixStream2(rand(i));
final long start = System.currentTimeMillis();
final MatrixStream2 mul = a.addComponents(b);
final long now = System.currentTimeMillis();
final double elapsed = (now - start);
return new Double[] { (double) i, elapsed };
}
private static double[][] rand(final int size) {
final double[][] rnd = new double[size][size];
for (int i = 0; i < size; i++) {
for (int j = 0; j < size; j++) {
rnd[i][j] = Math.random();
}
}
return rnd;
}
}
结果：
Classic Matrix size, Time (ms)
100.0,1.0
500.0,5.0
900.0,5.0
1300.0,43.0
1700.0,94.0
2100.0,26.0
2500.0,33.0
2900.0,46.0
3300.0,265.0
3700.0,71.0
4100.0,87.0
4500.0,380.0
4900.0,432.0
5300.0,215.0
5700.0,238.0
6100.0,577.0
6500.0,677.0
6900.0,609.0
7300.0,584.0
7700.0,592.0
Stream1, Time(ms)
100.0,86.0
500.0,13.0
900.0,9.0
1300.0,47.0
1700.0,92.0
2100.0,29.0
2500.0,33.0
2900.0,46.0
3300.0,253.0
3700.0,71.0
4100.0,90.0
4500.0,352.0
4900.0,373.0
5300.0,497.0
5700.0,485.0
6100.0,579.0
6500.0,711.0
6900.0,800.0
7300.0,780.0
7700.0,902.0
Stream2, Time(ms)
100.0,111.0
500.0,42.0
900.0,12.0
1300.0,54.0
1700.0,97.0
2100.0,110.0
2500.0,177.0
2900.0,71.0
3300.0,250.0
3700.0,106.0
4100.0,359.0
4500.0,143.0
4900.0,233.0
5300.0,261.0
5700.0,289.0
6100.0,406.0
6500.0,814.0
6900.0,830.0
7300.0,828.0
7700.0,911.0
我为了更好的比较制作了一个情节：
I made a plot for better comparison:
根本没有任何改善。哪个是缺陷？矩阵是否小（7700 x 7700）？大于此，它会炸毁我的计算机内存。
There's no improvement at all. Where's the flaw? Are the matrices to small (7700 x 7700)? Greater than this it blows up my computer memory.
推荐答案
一种方法是使用 Arrays.parallelSetAll ：
One way to do it would be by using Arrays.parallelSetAll:
int rows = a.length;
int cols = a[0].length;
double[][] res = new double[rows][cols];
Arrays.parallelSetAll(res, i -> {
Arrays.parallelSetAll(res[i], j -> a[i][j] + b[i][j]);
return res[i];
});
我不是100％肯定，但我认为内部调用 Arrays.parallelSetAll 可能不值得为每行的列生成内部并行化的开销。也许仅仅足以并行化每行的总和：
I'm not 100% sure, but I think the inner call to Arrays.parallelSetAll might not be worth the overhead of generating inner parallelization for each row's columns. Maybe it's just enough to parallelize the sum for each row only:
Arrays.parallelSetAll(res, i -> {
Arrays.setAll(res[i], j -> a[i][j] + b[i][j]);
return res[i];
});
无论如何，在将并行化添加到算法之前，您应该仔细测量，因为很多时候开销太大了它不值得使用它。
Anyway, you should measure carefully before adding parallelization to an algorithm, because many times the overhead is so big that it's not worth using it.
这篇关于Java和两个double [] []并行流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
