使用大型阵列-OutOfRam [英] Working with large arrays - OutOfRam
问题描述
我有一个算法，可以创建两个这样的二维数组:
I have an algorithm where I create two bi-dimensional arrays like this:
TYPE
TPtrMatrixLine = array of byte;
TCurMatrixLine = array of integer;
TPtrMatrix = array of TPtrMatrixLine;
TCurMatrix = array of TCurMatrixLine;
function x
var
PtrsMX: TPtrMatrix;
CurMx : TCurMatrix;
begin
{ Try to allocate RAM }
SetLength(PtrsMX, RowNr+1, ColNr+1);
SetLength(CurMx , RowNr+1, ColNr+1);
for all rows do
for all cols do
FillMatrixWithData; <------- CPU intensive task. It could take up to 10-20 min
end;
两个矩阵始终具有相同的维数.通常矩阵中只有2000行和2000列，但是有时它可以高达25000x6000，因此对于这两个矩阵，我都需要146.5 + 586.2 = 732.8MB的RAM.问题在于这两个块必须是连续的，因此在大多数情况下，即使在现代计算机上似乎没有500-600MB的可用RAM，我也会用光RAM.
The two matrices have always the same dimension. Usually there are only 2000 lines and 2000 columns in the matrix but sometimes it can go as high as 25000x6000 so for both matrices I need something like 146.5 + 586.2 = 732.8MB of RAM. The problem is that the two blocks need to be contiguous so in most cases, even if 500-600MB of free RAM doesn't seem much on a modern computer, I run out of RAM.
该算法会根据该单元的邻居为该单元填充数据.这些操作只是加法和减法.
The algorithm fills the cells of the array with data based on the neighbors of that cell. The operations are just additions and subtractions.
由于TCurMatrixLine使用整数存储数据，因此它占用大量内存或RAM.不幸的是，存储的值可能带有符号，因此我不能使用Word而不是整数.SmallInt太小(我的值大于SmallInt，但小于Word).我希望如果有其他方法可以实现此目的，则不需要增加太多开销，因为处理具有如此多行/列的矩阵已经花费了很多时间.换句话说，我希望减少内存需求不会增加处理时间.
The TCurMatrixLine is the one that takes a lot or RAM since it uses integers to store data. Unfortunately, values stored may have sign so I cannot use Word instead of integers. SmallInt is too small (my values are bigger than SmallInt, but smaller than Word). I hope that if there is any other way to implement this, it needs not to add a lot of overhead, since processing a matrix with so many lines/column already takes a lot of time. In other words I hope that decreasing memory requirements will not increase processing time.
有什么想法可以减少内存需求吗?[我使用Delphi 7]
Any idea how to decrease the memory requirements? [I use Delphi 7]
更新有人建议我数组的每一行应该是一个独立的一维数组.我根据需要创建任意多的行(数组)并将它们存储在TList中.听起来很好.显然，分配这样的小内存块不会有问题.但是，恐怕它将对速度产生巨大影响.我现在使用
Update Somebody suggested that each row of my array should be an independent uni-dimensional array. I create as many rows (arrays) as I need and store them in TList. Sound very good. Obviously there will be no problem allocation such small memory blocks. But I am afraid it will have a gigantic impact on speed. I use now
TCurMatrixLine = array of integer;
TCurMatrix = array of TCurMatrixLine;
因为它比 TCurMatrix =整数数组的数组快(因为数据在内存中的放置方式).因此，以独立的行中断数组可能会影响速度.
because it is faster than TCurMatrix= array of array of integer (because of the way data is placed in memory). So, breaking the array in independent lines may affect the speed.
推荐答案
使用有符号2字节整数的建议将对您有很大帮助.
The suggestion of using a signed 2 byte integer will greatly aid you.
另一种有用的策略是，通过向您的.dpr文件中添加 {$ SetPEFlags IMAGE_FILE_LARGE_ADDRESS_AWARE} ，将您的exe标记为 LARGE_ADDRESS_AWARE .仅当您在64位Windows上运行时，这才有帮助，并将地址空间从2GB增加到4GB.
Another useful tactic is to mark your exe as being LARGE_ADDRESS_AWARE by adding {$SetPEFlags IMAGE_FILE_LARGE_ADDRESS_AWARE} to your .dpr file. This will only help if you are running on 64 bit Windows and will increase your address space from 2GB to 4GB.
它可能无法在Delphi 7上运行(我记得您正在使用D7)，并且您必须使用FastMM，因为旧的Borland内存管理器与大地址空间不兼容.如果 $ SetPEFlags 不可用，您仍然可以使用 EDITBIN 标记该exe.
It may not work on Delphi 7 (I seem to recall you are using D7) and you must be using FastMM since the old Borland memory manager isn't compatible with large address space. If $SetPEFlags isn't available you can still mark the exe with EDITBIN.
如果仍然遇到困难，那么另一个技巧是分配较小的内存子块，并使用包装器类处理映射索引到适当的子块和其中的偏移量.您可以使用默认的index属性，以使其对调用代码透明.
If you still encounter difficulties then yet another trick is to do allocate smaller sub-blocks of memory and use a wrapper class to handle mapping indices to the appropriate sub-block and offset within. You can use a default index property to make this transparent to the calling code.
自然地，像这样的块分配方法会招致一些处理开销，但是如果遇到获取连续块的麻烦，这是最好的选择.
Naturally a block allocated approach like this does incur some processing overhead but it's your best bet if you are having troubles with getting contiguous blocks.
这篇关于使用大型阵列-OutOfRam的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
