在Matlab中可视化大型矩阵 [英] Visualizing a large matrix in matlab

问题描述

我有一个巨大的稀疏矩阵(1,000 x 1,000,000)，无法在matlab上加载(内存不足).

I have a huge sparse matrix (1,000 x 1,000,000) that I cannot load on matlab (not enough RAM).

我想对这个矩阵进行可视化处理，以了解其稀疏性和值的差异.

I want to visualize this matrix to have an idea of its sparsity and of the differences of the values.

由于内存限制，我要按以下步骤进行操作:

Because of the memory constraints, I want to proceed as follows:

1-将矩阵分为4个矩阵

1- Divide the matrix into 4 matrices

2-将每个矩阵加载到Matlab上并进行可视化处理，以便颜色使您对值(尤其是零)有所了解

2- Load each matrix on matlab and visualize it so that the colors give an idea of the values (and of the zeros particularly)

3-粘贴"我将获得的4张图像，以便对原始矩阵有一个整体的想法

3- "Stick" the 4 images I will get in order to have a global idea for the original matrix

(i)是否可以在matlab中加载矩阵的一部分"?

(i) Is it possible to load "part of a matrix" in matlab?

(ii)对于可视化工具，我了解间谍(和怀疑).但是，此功能只能使非零值的比例可视化.有没有添加颜色代码的方法?

(ii) For the visualization tool, I read about spy (and daspect). However, this function only enables to visualize the non-zero values indifferently of their scales. Is there a way to add a color code?

(iii)如何粘"图以制作图?

(iii) How can I "stick" plots in order to make one?

推荐答案

如果矩阵稀疏，则当前存储它(作为文本文件中的完整矩阵)的方法似乎效率很低，并且肯定会使很难将其加载到MATLAB中.但是，我怀疑只要它足够稀疏，它仍然可以作为稀疏矩阵引入到MATLAB中.

If your matrix is sparse, then it seems that the currently method of storing it (as a full matrix in a text file) is very inefficient, and certainly makes loading it into MATLAB very hard. However, I suspect that as long as it is sparse enough, it can still be leaded into MATLAB as a sparse matrix.

执行此操作的传统方法是一次全部加载，然后转换为稀疏表示.但是，在您的情况下，一次读入一行文本文件并即时将其转换为MATLAB稀疏矩阵是有意义的.

The traditional way of doing this would be to load it all in at once, then convert to sparse representation. In your case, however, it would make sense to read in the text file, one line at a time, and convert to a MATLAB sparse matrix on-the-fly.

您可以通过估计矩阵的稀疏性并使用它来查看是否可以将整个事物作为稀疏矩阵加载到MATLAB的内存中，从而找出是否可行.

You can find out if this is possible by estimating the sparsity of your matrix, and using this to see if the whole thing could be loaded into MATLAB's memory as a sparse matrix.

尝试类似:(未经测试的代码！)

Try something like: (untested code!)

% initialise sparse matrix
sparse_matrix = sparse(num_rows, num_cols);
row_num = 1;

fid = fopen(filename);

% read each line of text file in turn
while ~feof(fid)
    this_line = fscanf(fid, '%f');

    % add row to sparse matrix (note transpose, which I think is required)
    sparse_matrix(row_num, :) = this_line';
    row_num = row_num + 1;
end
fclose(fid)

% visualise using spy
spy(sparse_matrix)

---

可视化

关于可视化:可以通过像imagesc这样的工具可视化稀疏矩阵，但是我相信它可以在内部创建完整的矩阵–也许有人可以确认这是否成立.如果确实如此，那么它将导致您出现内存问题.

With regards to visualisation: visualising a sparse matrix like this via a tool like imagesc is possible, but I believe it may internally create the full matrix – maybe someone can confirm if this is true or not. If it does, then it's going to cause you memory problems.

所有spy真正在做的是在2D中绘制非零元素的位置.您可以相当容易地编写自己的间谍函数，根据每个位置的值，间谍函数可以具有不同的颜色或大小的点.有关某些示例，请参见此答案.

All spy is really doing is plotting in 2D the locations of the non-zero elements. You can fairly easily write your own spy function, which can have different coloured or sized points depending on the values at each location. See this answer for some examples.

保存稀疏矩阵

如上所述，将矩阵保存为方法的效率非常低–对于稀疏度为10％的矩阵，文本文件中约95％的值将为零或空格.我不知道这些数据是从哪里来的，但是如果您对它的创建有任何控制权(例如，它来自您编写的另一个程序)，则将非零元素仅保存为格式row_idx, col_idx, value.

As I say above, the method your matrix is saved as is pretty inefficient – for a matrix with 10% sparsity, around 95% of your text file will be a zero or a space. I don't know where this data has come from, but if you have any control over its creation (e.g. it comes from another program you have written) it would make much more sense to save only the non-zero elements in the format row_idx, col_idx, value.

然后，您可以使用 spconvert 进行导入稀疏矩阵直接.

You can then use spconvert to import the sparse matrix directly.

这篇关于在Matlab中可视化大型矩阵的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！