如何在将大数据表转换为json时解决内存不足错误 [英] How to Solve out of memory error when converting large datatable into json

问题描述

我通过sql查询填充数据集，数据集包含一个大数据表。

 DataSet dataSet1 = new DataSet（）; 
 SqlDataAdapter ndaGlobalClass = new SqlDataAdapter（Query，cn）; 
 ndaGlobalClass.SelectCommand.CommandTimeout = 0; 
 cn.Open（）; 
 ndaGlobalClass.Fill（dataSet1）; 
 cn.Close（）; 
 string s = JsonConvert.SerializeObject（dataSet1.Tables [0]）; 

查询返回大数据表时我通过序列化System.OutOfMemoryException'将它转换为json。我该如何解决这个问题？我需要序列化大型数据表，并且没有循环问题。

解决方案

如果您正在使用大型数据表并且内存不足，那很可能因为JSON字符串的大小对于.NET来说太大了 - 在.NET中的任何单个对象上都存在2GB的限制，并且由于JSON是基于文本的序列化，因此大表可能会超过该表，即使原始的数据表远小于此。



尝试实验：找出表中包含多少行，并修改查询以仅返回一半：SELECT TOP nnn应该这样做。然后看看你是否可以将其转换为JSON，如果是，那么生成的字符串有多大。这应该会让你知道这是否只是有点傻大小，你可能最好找到一种不同的方式来传输数据！ ：笑：

I Fill a dataset by sql query and the dataset contains a large datatable.

DataSet dataSet1 = new DataSet();
SqlDataAdapter ndaGlobalClass = new SqlDataAdapter(Query, cn);
ndaGlobalClass.SelectCommand.CommandTimeout = 0; 
cn.Open();
ndaGlobalClass.Fill(dataSet1);
cn.Close();
string s=JsonConvert.SerializeObject(dataSet1.Tables[0]);

the Query return a large datatable and when i convert it to json by serializing System.OutOfMemoryException' was thrown. How can i fix this problem? i need to serialize large datatable and there is no circular issue.

解决方案

If you are using a large data table and you are getting out of memory issues, it may well be that the size of the JSON string is just too big for .NET - there is a limit of 2GB on any single object in .NET, and since JSON is a text-based serialization a large table could well exceed that even if the "raw" data table is considerably less than that.

Try an experiment: find out how many rows the table holds, and modify your query to return only half that: SELECT TOP nnn should do it. Then see if you can convert that to JSON and if so how big the resulting string is. That should give you an idea if this is just getting a bit silly size wise, and you might be better off finding a different way to transfer the data! :laugh:

这篇关于如何在将大数据表转换为json时解决内存不足错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！