内存不足读取具有8GB RAM的300MB文件 [英] Out of Memory Reading 300MB file with 8GB RAM

问题描述

我的机器上有8GB的RAM，其中有大量可用空间.
尝试将300MB文件读入系统字符串会给我发出内存不足错误.为什么?有修补程序吗?我记得使用.NET通过中等大小的电子邮件通过电子邮件发送附件时也遇到类似的问题.

I have 8GB of RAM on my machine, with a significant amount free.
Trying to read 300MB file into a System String gives me an Out of Memory error. Why? Is there a hotfix? I remember a similar problem with emailing attachments over a moderate size with .NET.

System.IO.StreamReader myFile = new System.IO.StreamReader("C:\\Z1_struct.xml");
System.String File_As_String = myFile.ReadToEnd();
myFile.Close();

推荐答案

300 MB可能编码为UTF-8的XML-> 600 MB UTF-16字符串

ReadToEnd在内部使用StringBuilder，它返回sb.ToString().
仅此一项就导致至少1200 MB被分配在一个点上.还有一点缓冲和用于从UTF-8转换为UTF-16的内存.

如果在ASP.Net中执行此操作，则应用程序池通常具有默认的内存限制-通常会导致池被回收.

如果您在32位操作系统下运行，那么这8 GB根本无济于事，即使您在64位操作系统下运行，也必须确保该进程作为64位进程执行.在Visual Studio下运行它默认为32位执行...

我会尝试另一种处理300 MB xml文件的方法.

一个有效的基于SAX的方法可能会更好-我尚未测试过，但这也许值得一试:
建立声明性SAX框架:第1部分-简单的SAX到C#映射 [ ^ ]

最好的问候
Espen Harlinn

300 MB XML likely encoding UTF-8 -> 600 MB UTF-16 string

ReadToEnd uses a StringBuilder internally, it returns sb.ToString().
That alone results in at least 1200 MB being allocated at one point. There is also a bit of buffering and the memory used to convert from UTF-8 to UTF-16.

If you are doing this in ASP.Net the application pool has usually a default memory limit - usually resulting in the pool being recycled.

If you are running under a 32-bit os those 8 GB will not help at all, and even if you are running under a 64-bit os, you have to make sure that the process is executed as a 64-bit process. Running it under Visual Studio defaults to 32-bit execution ...

I would try another method of processing a 300 MB xml file.

A working SAX based approach would probably be better - I haven''t tested this, but it is perhaps worth a try:
Towards a Declarative SAX Framework : Part 1 - A Simple SAX-to-C#-Mapping[^]

Best regards
Espen Harlinn

这 [ ^ ]可以为您提供帮助.

This[^] could help you.

这篇关于内存不足读取具有8GB RAM的300MB文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！