让AccumArray输出一个表 [英] Letting accumarray output a table
问题描述
accumarray使用两行索引来创建一个矩阵，在有效索引对的位置上具有元素，该元素具有由指定函数分配的值，例如:
A = [11:20];
B = flipud([11:20]);
C = 1:10;
datamatrix = accumarray([A B],C);
这样，datamatrix将成为具有值的20x20矩阵.但是，如果A和B的值很大，则将导致矩阵大部分为空，并且在远角有少量数据.为了避免这种情况，可以将accumarray设置为issparse:
sparsedatamatrix = accumarray([A B],C,[],@sum,[],true);
如果min(A)和/或min(B)很大，这将节省大量内存.
但是，我的问题是，我有一个Mx7矩阵和一个M~1e8，我希望基于前两个列的索引以及该列的标准偏差来收集第3列到第7列的均值第三列也基于第三列:
result = accumarray([data(:,1) data(:,2)],data(:,3),[],@std);
我想将其保存回表中，结构为[X Y Z std R G B I]，其中X和Y是索引，Z是该像素的平均高度，R，G， B和I是每个像素的平均值(颜色和强度)，std是高度(即粗糙度)的标准偏差.在这种情况下使用issparse并没有帮助，因为我使用repmat转换了accumarray的矩阵.
此代码的重点是从点云中估算一块土地的高度，粗糙度，颜色和强度.我将X和Y的坐标四舍五入以创建一个网格，现在需要每个网格单元的平均值，但要以表格"形式输出(不是MATLAB数据类型，而是不是默认矩阵输出的2D数组). /p>
因此，我们要总结一下这个问题:
accumarray或类似函数是否可以在没有中间(可能非常大)矩阵的情况下输出此表?
以下代码:
Xmax = max(Originaldata(:,1));
Ymax = max(Originaldata(:,2));
X_avg_grid=(Edgelength:Edgelength:Xmax*Edgelength)+Xorig;
TestSet = zeros(Xmax*Ymax,9);
xx = [1:length(X_avg_grid)]'; %#ok<*NBRAK>
TestSet(:,1) = repmat(xx,Ymax,1);
ll = 0:Xmax:Xmax*Ymax;
for jj = 1:Ymax
TestSet(ll(jj)+1:ll(jj+1),2) = jj;
end
for ll = 1:7
if ll == 2
tempdat = accumarray([Originaldata(:,1) Originaldata(:,2)],Originaldata(:,3),[],@std);
tempdat = reshape(tempdat,[],1);
TestSet(:,ll+2) = tempdat;
elseif ll == 7
tempdat = accumarray([Originaldata(:,1) Originaldata(:,2)],1);
tempdat = reshape(tempdat,[],1);
TestSet(:,ll+2) = tempdat;
elseif ll == 1
tempdat = accumarray([Originaldata(:,1) Originaldata(:,2)],Originaldata(:,3),[],@mean);
tempdat = reshape(tempdat,[],1);
TestSet(:,ll+2) = tempdat;
else
tempdat = accumarray([Originaldata(:,1) Originaldata(:,2)],Originaldata(:,ll+1),[],@mean);
tempdat = reshape(tempdat,[],1);
TestSet(:,ll+2) = tempdat;
end
end
TestSet = TestSet(~(TestSet(:,9)==0),:);
这里的第九列只是每个单元格的点数.
Originaldata =
19 36 2.20500360107422 31488 31488 31488 31611
20 37 2.26400360107422 33792 33792 34304 33924
20 37 2.20000360107422 33536 33536 34048 33667
19 36 2.20500360107422 34560 34560 34560 34695
20 36 2.23300360107422 32512 32512 33024 32639
21 38 2.22000360107422 31744 31488 33024 31611
21 37 2.20400360107422 32512 32768 33792 32896
21 37 2.24800360107422 29696 29440 30720 29555
21 38 2.34800360107422 32768 32768 32768 32639
21 37 2.23000360107422 33024 33024 33536 33153
因此，对同一X，Y上的所有点(例如，[19 36]或[21 37])进行平均(高度，RGB，强度按此顺序)，并且第三列中的值也需要标准偏差:
Result =
19 36 2.2050036 0.00 33024 33024 33024 33153
21 37 2.227336934 0.02212088 31744 31744 32682.66 31868
以此类推，以获取其余数据.
我将代码更新为最新版本.这大大减少了内存开销，因为该函数现在一次又一次创建网格，而不是一次创建所有网格.但是，代码是并行运行的，因此仍然创建了八个同时的网格，因此仍然可以找到解决方案.
使用线性索引和2D稀疏矩阵的解决方案的草图
lind = Originaldata(:,1) + max( Originaldata(:,1) ) * ( Originaldata(:,2) - 1 );
daccum(7,:) = accumarray( lind, 1, [], @sum, [], true ); %// start with last one to pre-allocate all daccum
daccum(1,:) = accumarray( lind, Originaldata(:,3), [], @mean, [], true );
daccum(2,:) = accumarray( lind, Originaldata(:,3), [], @std, [], true );
daccum(3,:) = accumarray( lind, Originaldata(:,4), [], @mean, [], true );
daccum(4,:) = accumarray( lind, Originaldata(:,5), [], @mean, [], true );
daccum(5,:) = accumarray( lind, Originaldata(:,6), [], @mean, [], true );
daccum(6,:) = accumarray( lind, Originaldata(:,7), [], @mean, [], true );
现在您只能得到所需的东西
inter = [Originaldata(:,1), Originaldata(:,2), full( daccum(:,lind) )' ];
accumarray uses two rows of indices to create a matrix with elements on the location of valid index pairs with a value assigned by the specified function, e.g.:
A = [11:20];
B = flipud([11:20]);
C = 1:10;
datamatrix = accumarray([A B],C);
This way datamatrix will be a 20x20 matrix with values. If the values of A and B however are very large, this will result in a mostly empty matrix, with a small batch of data in the far corner. To circumvent this, one might set accumarray to issparse:
sparsedatamatrix = accumarray([A B],C,[],@sum,[],true);
This will save a lot of memory in case min(A) and/or min(B) is/are very large.
My problem, however, is that I have a Mx7 matrix, with M~1e8, on which I want to collect the means of columns three through seven based upon indexing in the first two columns and the standard deviation of the third column based upon the third as well:
result = accumarray([data(:,1) data(:,2)],data(:,3),[],@std);
I want to save this back into a table, structured as [X Y Z std R G B I], where X and Y are the indices, Z is the average height of that pixel,R, G, B and I are mean values (colour and intensity) per pixel and std is the standard deviation of heights (i.e. the roughness). Using the issparse in this case does not help, since I transform my matrices resulting from accumarray using repmat.
The point of this code is to estimate the height, roughness, colour and intensity of a piece of land from a point cloud. I rounded the coordinates in X and Y to create a grid and now need those average values per grid cell, but output as a "table" (not the MATLAB data type, but a 2D array which is not the default matrix output).
So, to conclude with the question:
Is there a way for accumarray or a similar function to output this table without intermediate (potentially very large) matrix?
Code below:
Xmax = max(Originaldata(:,1));
Ymax = max(Originaldata(:,2));
X_avg_grid=(Edgelength:Edgelength:Xmax*Edgelength)+Xorig;
TestSet = zeros(Xmax*Ymax,9);
xx = [1:length(X_avg_grid)]'; %#ok<*NBRAK>
TestSet(:,1) = repmat(xx,Ymax,1);
ll = 0:Xmax:Xmax*Ymax;
for jj = 1:Ymax
TestSet(ll(jj)+1:ll(jj+1),2) = jj;
end
for ll = 1:7
if ll == 2
tempdat = accumarray([Originaldata(:,1) Originaldata(:,2)],Originaldata(:,3),[],@std);
tempdat = reshape(tempdat,[],1);
TestSet(:,ll+2) = tempdat;
elseif ll == 7
tempdat = accumarray([Originaldata(:,1) Originaldata(:,2)],1);
tempdat = reshape(tempdat,[],1);
TestSet(:,ll+2) = tempdat;
elseif ll == 1
tempdat = accumarray([Originaldata(:,1) Originaldata(:,2)],Originaldata(:,3),[],@mean);
tempdat = reshape(tempdat,[],1);
TestSet(:,ll+2) = tempdat;
else
tempdat = accumarray([Originaldata(:,1) Originaldata(:,2)],Originaldata(:,ll+1),[],@mean);
tempdat = reshape(tempdat,[],1);
TestSet(:,ll+2) = tempdat;
end
end
TestSet = TestSet(~(TestSet(:,9)==0),:);
The ninth column here is just the amount of points per cell.
Originaldata =
19 36 2.20500360107422 31488 31488 31488 31611
20 37 2.26400360107422 33792 33792 34304 33924
20 37 2.20000360107422 33536 33536 34048 33667
19 36 2.20500360107422 34560 34560 34560 34695
20 36 2.23300360107422 32512 32512 33024 32639
21 38 2.22000360107422 31744 31488 33024 31611
21 37 2.20400360107422 32512 32768 33792 32896
21 37 2.24800360107422 29696 29440 30720 29555
21 38 2.34800360107422 32768 32768 32768 32639
21 37 2.23000360107422 33024 33024 33536 33153
So all points on the same X,Y (e.g. [19 36] or [21 37]) are averaged (height, RGB, intensity in that order) and of the values in the third column the standard deviation is also desired:
Result =
19 36 2.2050036 0.00 33024 33024 33024 33153
21 37 2.227336934 0.02212088 31744 31744 32682.66 31868
and so forth for the rest of the data.
I updated my code to the latest version I have. This reduced memory overhead quite a bit, as the function now creates the grids one after another as opposed to all at once. However, the code is running in parallel so there are still eight simultaneous grids created, so a solution would still be appreciated.
A sketch of a solution using linear indices and 2D sparse matrix
lind = Originaldata(:,1) + max( Originaldata(:,1) ) * ( Originaldata(:,2) - 1 );
daccum(7,:) = accumarray( lind, 1, [], @sum, [], true ); %// start with last one to pre-allocate all daccum
daccum(1,:) = accumarray( lind, Originaldata(:,3), [], @mean, [], true );
daccum(2,:) = accumarray( lind, Originaldata(:,3), [], @std, [], true );
daccum(3,:) = accumarray( lind, Originaldata(:,4), [], @mean, [], true );
daccum(4,:) = accumarray( lind, Originaldata(:,5), [], @mean, [], true );
daccum(5,:) = accumarray( lind, Originaldata(:,6), [], @mean, [], true );
daccum(6,:) = accumarray( lind, Originaldata(:,7), [], @mean, [], true );
Now you can get only what you need
inter = [Originaldata(:,1), Originaldata(:,2), full( daccum(:,lind) )' ];
这篇关于让AccumArray输出一个表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
