如何在MATLAB中加载大文件(〜150MB)? [英] How can I load large files (~150MB) in MATLAB?
问题描述
从版本R2011b(ver.7.13)开始,有一个新的对象 matlab.io.MatFile 与 MATFILE 作为构造函数。它允许加载和保存MAT文件中的部分变量。请参阅文档了解更多详情。这里是一个简单的例子来阅读一个矩阵的一部分:
pre $ mat $ = matfile(filename);
a = matObj.a(100:500,200:600);
如果您的原始文件不是MAT文件,而是一些文本文件,您可以部分读取使用 matfile
将这些部分保存到MAT文件中的相同变量中,以便以后访问。只要记得在构造函数中设置 Writable
属性为true就可以了。
假设你的文本文件是制表符分隔的,只有数字,这里是一个示例脚本,通过块读取数据并将其保存到MAT文件中:
blocksize = 100;
startrow = 0;
filename ='test.mat';
matObj = matfile(文件名,'Writable',true);
while true
try
a = dlmread(filename,'\t',startrow,0); %#取决于你的文件格式
startrow = startrow + blocksize;
matObj.a(startrow +(1:blocksize),:) = a;
catch
break
end
end
I现在没有最新版本来测试,但希望它能正常工作。
I have a large MATLAB file (150MB) in matrix form (i.e. 4070x4070). I need to work on this file in MATLAB but I can't seem to load this file. I am getting an "out of memory" error. Is there any other way I can load this size of file? I am using a 32bit processor and have 2GB of RAM. Please help me, I am getting exhausted from dealing with this problem.
Starting from release R2011b (ver.7.13) there is a new object matlab.io.MatFile with MATFILE as a constructor. It allows to load and save parts of variables in MAT-files. See the documentation for more details. Here is a simple example to read part of a matrix:
matObj = matfile(filename);
a = matObj.a(100:500, 200:600);
If your original file is not a MAT file, but some text file, you can read it partially and use matfile
to save those parts to the same variable in a MAT file for later access. Just remember to set Writable
property to true in the constructor.
Assuming your text file is tab-delimited and contains only numbers, here is a sample script to read the data by blocks and save them to MAT file:
blocksize = 100;
startrow = 0;
filename = 'test.mat';
matObj = matfile(filename,'Writable',true);
while true
try
a = dlmread(filename,'\t',startrow,0); %# depends on your file format
startrow = startrow + blocksize;
matObj.a(startrow+(1:blocksize),:) = a;
catch
break
end
end
I don't have the latest release now to test, but hope it should work.
这篇关于如何在MATLAB中加载大文件(〜150MB)?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!