Matlab中大型非稀疏矩阵的有效运算 [英] Efficient operations of big non-sparse matrices in Matlab

问题描述

我需要在Matlab中的3维大稀疏矩阵中进行运算.使用纯矢量化会产生较高的计算时间.因此，我尝试将操作分成10个块，然后解析结果. 当我看到纯向量化不能很好地缩放数据大小时，我感到惊讶，如下图所示.

I need to operate in big 3-dim non-sparse matrices in Matlab. Using pure vectorization gives a high computation time. So, I have tried to split the operations into 10 blocks and then parse the results. I got surprised when I saw the the pure vectorization does not scale very well with the data size as presented in the following figure.

我提供了这两种方法的示例.

I include an example of the two approaches.

% Parameters:
M = 1e6;  N = 50;  L = 4;  K = 10;

% Method 1: Pure vectorization
mat1 = randi(L,[M,N,L]);
mat2 = repmat(permute(1:L,[3 1 2]),M,N);
result1 = nnz(mat1>mat2)./(M+N+L);

% Method 2: Split computations
result2 = 0;
for ii=1:K
    mat1 = randi(L,[M/K,N,L]);
    mat2 = repmat(permute(1:L,[3 1 2]),M/K,N);
    result2 = result2 + nnz(mat1>mat2);
end
result2 = result2/(M+N+L);

因此，我想知道是否还有其他方法可以使Matlab中的大型矩阵运算更有效.我知道这是一个相当广泛的问题，但我会冒险的:)

Hence, I wonder if there is any other approach that makes big matrix operations in Matlab more efficient. I know it is a quite broad question, but I will take the risk :)

使用@Shai的实现

% Method 3
mat3 = randi(L,[M,N,L]);
result3 = nnz(bsxfun( @gt, mat3, permute( 1:L, [3 1 2] ) ))./(M+N+L);

时间是:

推荐答案

为什么repmat而不是bsxfun?

result = nnz(bsxfun( @gt, mat1, permute( 1:L, [3 1 2] ) ))./(M+N+L);

似乎您已经用完RAM，并且操作系统开始为交换非常大的矩阵分配空间.内存交换始终是非常耗时的操作，并且随着所需内存量的增加而变得更糟.
我相信您正在目睹 th撞.

It seems like you are using up your RAM and the OS starts to allocate room in swap for the very large matrics. Memory swapping is always a very time consuming operation and it gets worse as the amount of memory you require increases.
I believe you are witnessing thrashing.

这篇关于Matlab中大型非稀疏矩阵的有效运算的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！