限制RAM的使用. (C#.NET) [英] Limiting the use of RAM. (C# .NET)

问题描述

有大约100Mb的巨大文件. 我想将它们加载到内存(RAM)中，进行处理并保存在某个地方.

There are huge files about 100Mb. I want to load them into memory (RAM), process and save somewhere.

同时，我希望内存使用量存在限制.例如，对于我的应用程序而言，100Mb不会使用比该内存限制更多的内存.如果超出限制，则文件将被处理.

At the same time I want that a limit of memory usage exists. Example, 100Mb, to my app don't use more then this memory limit. If the limit is exceeded the file is processed parts.

我对此的理解:

var line = file.ReadLine();
var allowed = true;

while( allowed && line != null ) 
{
   var newObject = new SomeObject( line );
   list.add( newObject );

   // Checking the memory
   allowed = CheckUsedMemory(); 

   line = file.ReadLine()
} 

如何限制RAM的使用? 如何实现CheckUsedMemory方法? 谢谢.

How to limit the use of RAM? How to implement the CheckUsedMemory method? Thank you.

UPD

谢谢大家的好建议.

推荐答案

首先，感谢您了解内存消耗.如果只有更多的程序员这么体贴的话.

First, thanks for being aware of your memory consumption. If only more programmers were so considerate..

第二，我不会打扰:也许用户希望您的应用程序尽可能快地运行，并愿意刻录8000兆的内存，以使结果快5％.让他们. :)

Second, I wouldn't bother: perhaps the user wants your application to run as fast as possible and is willing to burn 8000 megs of memory to get results 5% faster. Let them. :)

但是，如果您在此过程中强制进行更多的磁盘访问，则人为地限制应用程序占用的内存量可能会大大增加处理时间.如果有人在内存受限的系统上运行，则他们很可能已经有磁盘流量可用于交换-如果您在真正完成内存处理之前人为地转储内存，那么只会为磁盘IO做出更大的贡献，交换方式.让操作系统处理这种情况.

But, artificially limiting the amount of memory your application takes may drastically increase processing time, if you force more disk-accesses in the process. If someone is running on a memory-constrained system, they are liable to already have disk traffic for swapping -- if you are artificially dumping memory before you're really finished with it, you're only contributing further to disk IO, getting in the way of the swapping. Let the OS handle this situation.

最后，您在此处编写的访问模式(顺序，一次一行)非常常见，并且.NET设计师无疑已经投入了大量精力从此模式到最低的内存使用量.将对象部分地添加到内部树中是一个不错的主意，但是很少有应用程序可以真正从中受益. (合并排序是一种出色的应用程序，它受益于部分处理.)

And lastly, the access pattern you've written here (sequential, line-at-a-time) is very common, and doubtless the .NET designers have put huge amounts of effort into getting memory usage from this pattern to the bare minimum. Adding objects to your internal trees in parts is a nice idea, but very few applications can really benefit from this. (Merge sorting is one excellent application that benefits greatly from partial processing.)

根据对完成的对象列表的处理方式，一次使用整个列表可能无法改善.或者，将其分解可能会大大受益. (如果地图精简很好地描述了您的数据处理问题，那么也许您可以从破坏问题中受益分开.

Depending upon what you're doing with your finished list of objects, you might not be able to improve upon working with the entire list at once. OR, you might benefit greatly from breaking it apart. (If Map Reduce describes your data processing problem well, then maybe you would benefit from breaking things apart.)

无论如何，我都会以内存"为基准来决定何时拆分处理，这有点让人不快:我宁愿使用"1000行输入"或十层嵌套"或运行了五分钟的机床"或其他基于输入的信息，而不是消耗内存的次要影响.

In any event, I'd be a little leery of using "memory" as the benchmark for deciding when to break apart processing: I'd rather use "1000 lines of input" or "ten levels of nesting" or "ran machine tools for five minutes" or something that is based on the input, rather than the secondary effect of memory consumed.

这篇关于限制RAM的使用. (C#.NET)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！