拆分大小为890 MB的xml文件时出现内存不足异常 [英] Out of memory exception while splitting xml file of size 890 MB
问题描述
嗨
请帮我解决我的错误，
i必须拆分基于节点/元素的数量（动作节点计数）将大xml文件转换为小的xml文件。当我这样做时，我得到内存不足。
示例Xml格式如下所示，但我的原始文件大小为890 MB从下面有2,90,973个动作节点。
< < span class =code-leadattribute> Parent ExternalID = CombinedTransactionsTest GeneratedDate = 2013-03-21 >
< 操作 ActionID = 4010 ActionType = Transaction.Add >
< 事务 EnteredDate = 2013-03-14 PostedDate = 2013-03-14 PostedBy = CONV >
< 调整 AccountID = S00000006 / >
< Taxlot AccountID = S00000006 SourceAccountID = S00000006 < span class =code-keyword> / >
< Parcel AccountID = S00000006 / >
< ParcelUnit AccountID = S00000006 / >
< / Transaction >
< ; / Action >
< 操作 ActionID = 4011 ActionType = Transaction.Add >
< 交易 EnteredDate = 2013-03-14 已发布日期 = 2013-03-14 PostedBy = CONV >
< 调整 AccountID = S00000006 / >
< Taxlot AccountID = S00000006 / >
< Parcel AccountID = S00000006 / >
< ParcelUnit AccountID = S00000006 / >
< / Transaction >
< / Action >
< / Parent > Hi 这篇关于拆分大小为890 MB的xml文件时出现内存不足异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
从前端（UI），用户能够指定多少动作< action>节点应该在每个拆分文件中。例如，如果我为每个文件指定10个动作，那么我需要每个文件有10个动作。
为了完成上述要求，我编写了如下代码。
代码
<前lang =cs> 如果（文件.Exists（txtConvXmlFilePath.Text））
{
bool exception = false ;
string FileName = Path.GetFileNameWithoutExtension（txtConvXmlFilePath.Text）;
if （！Directory.Exists（ D：\\XmlSplitFiles \\））
Directory.CreateDirectory（ d：\\XmlSplitFiles\\）;
var xDoc = XDocument.Load（txtConvXmlFilePath.Text）;
var xmls = xDoc.Root.Elements（）。ToArray（）;
int fileCount = 1 ;
var file = File.CreateText（ D：\\XmlSplitFiles \\ + FileName + fileCount.ToString（）+ .XML）;
StringBuilder sb = new StringBuilder（）;
sb.Append（ <）。附加（xDoc.Root.Name） ;
if （xDoc.Root.HasAttributes）
{
sb.Append（ ）;
var attval = xDoc.Root.Attribute（ 外部编号）;
if （attval！= null ）
sb.Append（attval）。追加（ ）;
var attval2 = xDoc.Root.Attribute（ GeneratedDate）;
if （attval2！= null ）
sb.Append（attval2）。追加（ ）;
sb.Append（ ）。追加（ >）;
}
int batchCount = Int32 .Parse（txtActionCount.Text.ToString（ ））;
file.Write（sb.ToString（）+ Environment.NewLine）;
for （ int i = 0 ; i < xmls.Length; i ++）
{
file.Write（xmls [i] .ToString（ ）+ Environment.NewLine）;
if （i == batchCount）
{
file.Write（Environment.NewLine + < / Parent>）;
file.Flush（）;
file.Close（）;
batchCount = batchCount + Int32 .Parse（txtActionCount.Text.ToString（））;
fileCount ++;
file = File.CreateText（ D：\\XmlSplitFiles \\ + FileName + fileCount.ToString（）+ 。xml）;
file.Write（sb.ToString（）+ Environment.NewLine）;
}
else if （i == xmls.Length - 1 ）
{
file.Write（Environment.NewLine + < /父母与gt; 中）;
file.Flush（）;
file.Close（）;
file = null ;
}
}
MessageBox.Show（ 拆分文件是存储在 + D：\\XmlSplitFiles \\） ;
}
来自上面的代码，
行
var xDoc = XDocument.Load （txtConvXmlFilePath.Text）; 是我收到错误的地方。
谢谢
请回答这个问题尽快。 将大型XML文件拆分成小文件 [ ^ ]
检查这些
如何在不加载XML文件的情况下打开大型XML文件？ [ ^ ]
解析/加载/搜索大小约1GB的XML文档 [ ^ ]
Please help me out what i am doing wrong,
i have to split a big xml file into small xml files based on the count of the nodes/element(action node count). While i am doing this i am getting Out of memory exception.
Sample Xml format is shown below but my Original file size is 890 MB with 2,90,973 actions nodes from below.
<Parent ExternalID="CombinedTransactionsTest" GeneratedDate="2013-03-21">
<Action ActionID="4010" ActionType="Transaction.Add" >
<Transaction EnteredDate="2013-03-14" PostedDate="2013-03-14" PostedBy="CONV">
<Adjustment AccountID="S00000006" />
<Taxlot AccountID="S00000006" SourceAccountID="S00000006" />
<Parcel AccountID="S00000006" />
<ParcelUnit AccountID="S00000006" />
</Transaction>
</Action>
<Action ActionID="4011" ActionType="Transaction.Add" >
<Transaction EnteredDate="2013-03-14" PostedDate="2013-03-14" PostedBy="CONV">
<Adjustment AccountID="S00000006" />
<Taxlot AccountID="S00000006"/>
<Parcel AccountID="S00000006"/>
<ParcelUnit AccountID="S00000006" />
</Transaction>
</Action>
</Parent>
From front end(UI), the user is able to specify how many action <action> nodes should be in each split-ted files. For example, if i specify 10 actions per file then i need to have 10 actions per file.
To do the above requirement, i have written the code as below.
Code
if (File.Exists(txtConvXmlFilePath.Text))
{
bool exception = false;
string FileName = Path.GetFileNameWithoutExtension(txtConvXmlFilePath.Text);
if (!Directory.Exists("D:\\XmlSplitFiles\\"))
Directory.CreateDirectory("D:\\XmlSplitFiles\\");
var xDoc = XDocument.Load(txtConvXmlFilePath.Text);
var xmls = xDoc.Root.Elements().ToArray();
int fileCount = 1;
var file = File.CreateText("D:\\XmlSplitFiles\\" + FileName + fileCount.ToString() + ".xml");
StringBuilder sb = new StringBuilder();
sb.Append("<").Append(xDoc.Root.Name);
if (xDoc.Root.HasAttributes)
{
sb.Append(" ");
var attval = xDoc.Root.Attribute("ExternalID");
if (attval != null)
sb.Append(attval).Append(" ");
var attval2 = xDoc.Root.Attribute("GeneratedDate");
if (attval2 != null)
sb.Append(attval2).Append(" ");
sb.Append(" ").Append(">");
}
int batchCount = Int32.Parse(txtActionCount.Text.ToString());
file.Write(sb.ToString() + Environment.NewLine);
for (int i = 0; i < xmls.Length; i++)
{
file.Write(xmls[i].ToString() + Environment.NewLine);
if (i == batchCount)
{
file.Write(Environment.NewLine + "</Parent>");
file.Flush();
file.Close();
batchCount = batchCount + Int32.Parse(txtActionCount.Text.ToString());
fileCount++;
file = File.CreateText("D:\\XmlSplitFiles\\"+ FileName + fileCount.ToString() + ".xml");
file.Write(sb.ToString() + Environment.NewLine);
}
else if (i == xmls.Length - 1)
{
file.Write(Environment.NewLine + "</Parent>");
file.Flush();
file.Close();
file = null;
}
}
MessageBox.Show("Split files are stored at " + "D:\\XmlSplitFiles\\");
}
From the above code,
Line
var xDoc = XDocument.Load(txtConvXmlFilePath.Text); is the place where i am getting error.
Thank you
Please answer this as soon as possible.Split large XML files into small files[^]
Check these
How to Open Large XML files without Loading the XML Files?[^]
Parsing/Loading/Searching XML Document of Size ~ 1GB[^]
