在C＃DataTable中存储数百万行时避免内存问题 [英] Avoid Memory Issues while storing millions of rows in C# DataTable

问题描述

我有一个Windows应用程序，可以处理目录中大约500k个文档，同时将一些信息存储在两个本地DataTable中，稍后将在应用程序中使用。这适用于小输入，例如100k文档。但是，当每个DataTable必须存储超过百万行时，应用程序占用了所有内存。

所以，请告诉我，如果有比.NET中的datatable更好的结构，那将是能够存储行和列的大量数据。
在这方面的任何帮助，想法和建议将不胜感激。

谢谢，
Jayanthi

Hi,

I have a windows application that processes around 500k documents in a directory, while storing some information in a two local DataTables, which will be used later in the application.
This works fine for small input, something like 100k documents. But, the application is eating up all the memory when each DataTable has to store more than million rows.

So, kindly let me know if there is any better structure than datatable in .NET, that will be able to store huge data with rows and columns.
Any help, ideas, suggestions in this regard will be greatly appreciated.

Thanks,
Jayanthi

推荐答案

您可以创建一个自定义类或结构来存储您的信息，并将其保存在List< T>中。这比DataTable的开销要小一些。
但是，在某些时候，将数十万条信息保存在内存中只需要大量内存。如果你想减少这种情况，可以存储更少的信息，或者使用一种存储机制，例如数据库，它允许你从内存中卸载它。

You could make a custom class or struct to store your information, and just keep it in a List<T>.  This will have somewhat less overhead than a DataTable.

However, at some point, keeping hundreds of thousands of pieces of information in memory is just going to require a lot of memory.  If you want to cut this back, store less information, or use a storage mechanism such as a database that will allow you to unload it from memory.

这篇关于在C＃DataTable中存储数百万行时避免内存问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！