如何在c＃中读取大型XML文件 [英] how to read large XML file in c#

问题描述

我有一个6 GB大小的非常大的XML文件我想读它并将其输出写入多个文件。

我目前正在使用XmlTextReader并且需要30分钟。

请建议一些减少处理时间的方法。

I have a very large XML file of 6 GB size i want to read it and write its output in multiple files .
I am currently using XmlTextReader and its taking 30 mins .
Please suggest some ways to reduce the processing time .

推荐答案

你的问题的标签提示正确的方法：使用 XmlReader 。它对所消耗的内存资源没有限制，因为它不会将任何内容解析为内存。我认为与这种方法相比，没有办法显着提高处理速度。



与此同时，我不知道还有什么您在解析期间执行此操作，但使用读者类除外。你总是要做别的事情，这个别的东西可能会很慢。显然，我无法访问您的硬盘驱动器，告诉您瓶颈在哪里。我可以建议的是：使用一些性能分析器来研究问题。请参阅：

https://msdn.microsoft.com/en-US /library/ms182372.aspx [ ^ ]，

https://msdn.microsoft.com/en -us / library / z9z62c29.aspx [ ^ ]。

-SA

The tags of your questions suggest correct approach: use XmlReader. It presents no limitations to the memory resources consumed, because it does not parse anything to memory. I think there is no a way to significantly increase the speed of processing compared to this approach.

At the same time, I don't know what else you do during parsing, except the use of the reader class. You always have to do something else, and this "something else" can be slow. Apparently, I have no access to your hard drive to tell you where the bottleneck is. All I can advise is: use some performance profiler to research the problem. Please see:
https://msdn.microsoft.com/en-US/library/ms182372.aspx[^],
https://msdn.microsoft.com/en-us/library/z9z62c29.aspx[^].

—SA

这是我的代码可以改进它。

this is my code can i improve it .

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            ReadEJFile("E:\\a.txt");
        }

        private static void ReadEJFile(string filename)
        {
            Stopwatch sw = Stopwatch.StartNew();
            Console.WriteLine(sw.Elapsed.TotalMilliseconds);
            XmlTextReader reader = new XmlTextReader(filename);
            reader.WhitespaceHandling = WhitespaceHandling.None;
            XDocument doc;

            reader.ReadToFollowing("Root");

            while (reader.Read())
            {
                switch (reader.NodeType)
                {
                    case XmlNodeType.Element:
                        if (reader.Name.Equals("journalEntry"))
                        {
                            doc = XDocument.Load(reader.ReadSubtree());
                            reader.ReadToDescendant("journalEntry");
                            var terminalId = doc.Descendants().Where(n => n.Name == "terminalIdentifier").SingleOrDefault().Value;
                            Stream xmlFile = new FileStream("E:\\Output\\" + terminalId + ".txt",FileMode.Append,FileAccess.Write);
                            XmlTextWriter textWriter = new XmlTextWriter(xmlFile, Encoding.Default);
                            doc.Save(textWriter);
                            textWriter.Close();

                        }
                       break;
                    
                }
                Console.WriteLine(sw.Elapsed.TotalMilliseconds);
                Console.WriteLine("inside");
            }          
        }
    }
}

这篇关于如何在c＃中读取大型XML文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！