如何在vb.net中加载内存中的大型xml [英] How to load large size xml in memory in vb.net

问题描述

当xml文件大小超过250 MB时，它会因内存不足而失败。

我们使用以下代码。

When the xml file size is above 250 MB it fails with out of memory exception.
we are using the below code.

Dim Final as string
Final = File.ReadAllText(sFTPLogXml)
newFinal = Replace(Final, "&", "&")

请告诉我有没有其他方法来加载大尺寸的xml文件。



我尝试过：



我尝试过使用XmlTextReader和Stream Reader。

Please let me know is there any other way to load large size xml files.

What I have tried:

I have tried with XmlTextReader and Stream Reader.

推荐答案

如果你要做的就是替换&安培;与& amp; amp;然后你不需要一次阅读整个文件。



你只需要阅读可管理的块。



以下代码我没有经过测试，但应该给出你需要做些什么来限制使用大文件的影响。



一旦开始工作就可以获得块大小和基准测试的性能，这样你就可以在最佳时间内获得结果。

If all you're trying to do is replace & with & then you don't need to read the whole file at once.

You only need to read in manageable blocks.

The following code I haven't tested but should give you the gist of what you need to do to limit the impact of working with large files.

Once working have a play with the block size and bench mark performance so you can achieve your result in the best time.

string inputFile = "myFile.xml";
string tempFile = "tmp.xml";

//open input and output file streams
System.IO.FileStream input = System.IO.File.OpenRead(inputFile);
System.IO.FileStream output = System.IO.File.Open(tempFile, System.IO.FileMode.Create, System.IO.FileAccess.Write);

//set start point, end point and block size
long position = 0;
long blockSize = 200;
long fileSize = input.Length;

long outputPosition = 0;

byte[] buffer = new byte[blockSize];

bool atEnd = false;

//while not at the
while(!atEnd)
{

  //manage block size for end of file read
  long currentBlock = blockSize;

  if (position + blockSize > fileSize)
    currentBlock = fileSize - position;

  //read input block
  input.Read(buffer, (int)position, (int)currentBlock);

  //do text replace and generate output block
  byte[] outputBuffer = System.Text.Encoding.UTF8.GetBytes(System.Text.Encoding.UTF8.GetString(buffer).Replace("&", "&"));

  //write output block
  output.Write(outputBuffer, (int)outputPosition, outputBuffer.Length);

  //if end of file
  if (position >= fileSize) {
    atEnd = true;
  } else {
      //if not increment pointers
      outputPosition += outputBuffer.Length;
      position += currentBlock;
  }

}

//close files
input.Close();
output.Close();

//delete original
System.IO.File.Delete(inputFile);
//rename temp file
System.IO.File.Move(tempFile, inputFile);

这篇关于如何在vb.net中加载内存中的大型xml的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！