C＃体型硕大2暗淡阵列 [英] C# huge size 2-dim arrays
问题描述
我需要连续超过20000项申报在C＃中的WinForms方阵。
我读到的32位2GB .NET对象大小限制以及在64位操作系统同样如此。
以便我明白了一个简单的答案 - 是使用不安全的代码或单独的库建withing C ++编译器
I need to declare square matrices in C# WinForms with more than 20000 items in a row. I read about 2GB .Net object size limit in 32bit and also the same case in 64bit OS. So as I understood the single answer - is using unsafe code or separate library built withing C++ compiler.
对于我的问题是值得，因为USHORT [20000， 20000]越小则2GB，但实际上，我不能连分配的内存700MB。我的极限是650MB的，我不明白为什么 - 我有32位的WinXP有3GB内存。
我试图用Marshal.AllocHGlobal（700℃;< 20）。但它抛出OutOfMemoryException异常，GC.GetTotalMemory试图分配内存之前返回4.5MB
The problem for me is worth because ushort[20000,20000] is smaller then 2GB but actually I cannot allocate even 700MB of memory. My limit is 650MB and I don't understand why - I have 32bit WinXP with 3GB of memory. I tried to use Marshal.AllocHGlobal(700<<20) but it throws OutOfMemoryException, GC.GetTotalMemory returns 4.5MB before trying to allocate memory.
我发现只有很多人说使用不安全的代码，但我无法找到的例子如何声明堆2维数组（任何堆栈不能保证数据的如此巨大数额），以及如何使用指针它的工作。
是纯它的C ++不安全{}括号中的代码？
I found only that many people say use unsafe code but I cannot find example of how to declare 2-dim array in heap (any stack can't keep so huge amount of data) and how to work with it using pointers. Is it pure C++ code inside of unsafe{} brackets?
PS。请不要问我为什么需要如此庞大的阵列......但如果你想 - 我需要分析文本（如图书），并发现大量的索引。所以答案是 - 词之间的关系矩阵
PS. Please don't ask WHY I need so huge arrays... but if you want - I need to analyze texts (for example books) and found lot of indexes. So answer is - matrices of relations between words
编辑：可能有人请提供使用指针的矩阵在不安全的代码工作中的一小示例。我知道，在32位是不可能的分配更多的空间，但我在谷歌上搜索这样的例子花了很多时间，没有发现任何
Could somebody please provide a small example of working with matrices using pointers in unsafe code. I know that under 32bit it is impossible to allocate more space but I spent much time in googling such example and found NOTHING
推荐答案
我真是太高兴了！ :)最近我打得四处主要问题 - 试图用数据库来解决，但只发现这种方式是远远完美。矩阵[20000,20000]被实现为一个表。
即使只有创造超过400百万条记录需要正确设置索引的时间是我的电脑上1个小时左右。这不是我的关键。
然后我跑了算法与矩阵的工作（需要两倍加入相同的表！）之后，它的工作半个多小时就再没甚至足下。
之后，我明白，唯一的办法就是找到一种方式，只有和回C＃再次在内存这样的矩阵工作。
I'm so happy! :) Recently I played around subject problem - tried to resolve it using database but only found that this way is far to be perfect. Matrix [20000,20000] was implemented as single table. Even with properly set up indexes time required only to create more than 400 millions records is about 1 hour on my PC. It is not critical for me. Then I ran algorithm to work with that matrix (require twice to join the same table!) and after it worked more than half an hour it made no even single step. After that I understood that only way is to find a way to work with such matrix in memory only and back to C# again.
我创建试点应用测试内存分配过程，以确定确切位置分配过程中停止使用不同的结构。
I created pilot application to test memory allocation process and to determine where exactly allocation process stops using different structures.
正如我的第一篇说，可以使用2 - 昏暗的阵列只有约 650MB 下的32位WinXP的分配。使用Win7和64位编译也感到难过后
结果 - 小于 700MB
As was said in my first post it is possible to allocate using 2-dim arrays only about 650MB under 32bit WinXP. Results after using Win7 and 64bit compilation also were sad - less than 700MB.
我用交错数组[] []，而不是单2维数组的[，]和结果可以看到如下：
I used JAGGED ARRAYS [][] instead of single 2-dim array [,] and results you can see below:
编译发布模式的32位应用程序 - 32位的WinXP物理3GB。纪念品。 - 1.45GB
在Release模式编译为64位应用程序 - VM win7下64位2GB - 7.5GB
Compiled in Release mode as 32bit app - WinXP 32bit 3GB phys. mem. - 1.45GB Compiled in Release mode as 64bit app - Win7 64bit 2GB under VM - 7.5GB
- 我用于测试应用程序的源连接到这个职位。
我不能在这里找到如何安装源文件，以便仅仅描述设计的一部分，并把这里手工代码。
创建WinForms应用程序。上表
将这种contols默认名称：
1键，1的NumericUpDown和1个列表框
在.cs文件添加下面的代码并运行
--Sources of application which I used for testing are attached to this post. I cannot find here how to attach source files so just describe design part and put here manual code. Create WinForms application. Put on form such contols with default names: 1 button, 1 numericUpDown and 1 listbox In .cs file add next code and run.
private void button1_Click(object sender, EventArgs e)
{
//Log(string.Format("Memory used before collection: {0}", GC.GetTotalMemory(false)));
GC.Collect();
//Log(string.Format("Memory used after collection: {0}", GC.GetTotalMemory(true)));
listBox1.Items.Clear();
if (string.IsNullOrEmpty(numericUpDown1.Text )) {
Log("Enter integer value");
}else{
int val = (int) numericUpDown1.Value;
Log(TryAllocate(val));
}
}
/// <summary>
/// Memory Test method
/// </summary>
/// <param name="rowLen">in MB</param>
private IEnumerable<string> TryAllocate(int rowLen) {
var r = new List<string>();
r.Add ( string.Format("Allocating using jagged array with overall size (MB) = {0}", ((long)rowLen*rowLen*Marshal.SizeOf(typeof(int))) >> 20) );
try {
var ar = new int[rowLen][];
for (int i = 0; i < ar.Length; i++) {
try {
ar[i] = new int[rowLen];
}
catch (Exception e) {
r.Add ( string.Format("Unable to allocate memory on step {0}. Allocated {1} MB", i
, ((long)rowLen*i*Marshal.SizeOf(typeof(int))) >> 20 ));
break;
}
}
r.Add("Memory was successfully allocated");
}
catch (Exception e) {
r.Add(e.Message + e.StackTrace);
}
return r;
}
#region Logging
private void Log(string s) {
listBox1.Items.Add(s);
}
private void Log(IEnumerable<string> s)
{
if (s != null) {
foreach (var ss in s) {
listBox1.Items.Add ( ss );
}
}
}
#endregion
问题解决了我。伙计们，谢谢提前！
The problem is solved for me. Guys, thank you in advance!
这篇关于C＃体型硕大2暗淡阵列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
