有效地解析大型XML [英] Large XML Parsing Efficiently

问题描述

我需要解析大型XML文件并将数据保存到MS SQL DB表.显然是编写C#程序的一种方法.显然，这引起了性能问题.您知道处理大规模XML的任何最快，最有效的方法吗?

I need to parse large XML Files and save data to MS SQL DB Tables. One way obviously to write C# Program. Obviously this raise a question of performance. Do you know any fastest and efficient way to process large scale XML?

推荐答案

如果要追求C#解决方案，请查看XmlReader.这将使您仅转发流式访问您的XML文件.请注意仅前进部分.如果需要对子节点进行更复杂的操作，则最好结合使用XmlReader和XDocument，即使用XmlReader加载大文件，然后使用ReadSubtree()加载子树进入XDocuments.例如，如果您的文档是这样的:

If you want to pursue a C# solution, look into XmlReader. This will give you forward only streaming access to your XML file. Note the forward only part. If you need to do more complex manipulations for child nodes, you'd probably do well to use a combination of XmlReader and XDocument, i.e. loading the large file with an XmlReader and then using ReadSubtree() to load subtrees into XDocuments. For example, if your document is something like:

<root>
    <big-child-1>
        <grandchild-a>
            ...
        </grandchild-a>
        <grandchild-b>
            ...
        </grandchild-b>
    </big-child-1>
    <big-child-2>
        ... 
    </big-child-2>
</root>

您可能会执行以下操作:

You might do something like this:

XmlReader xr = XmlReader.Create("C:\\file.xml");\
xr.MoveToContent();

while (xr.Read())
{
    if (xr.Name == "grandchild-a")
    {
        XDocument xd = new XDocument(xr.ReadSubTree()); // now you have an XDocument with all the content under the grandchild-a node
    }
    else if (xr.Name == ...)
}

但是，您可以仅仅使用XmlReader的次数越多，性能就越高.

However, the more you can just use XmlReader, the more performant it'll be.

以下是一些文档:

• XmlReader: https: //msdn.microsoft.com/zh-CN/library/system.xml.xmlreader%28v=vs.110%29.aspx
• XDocument: https://msdn.microsoft.com/zh-CN/library/system.xml.linq.xdocument%28v=vs.110%29.aspx

• XmlReader: https://msdn.microsoft.com/en-us/library/system.xml.xmlreader%28v=vs.110%29.aspx
• XDocument: https://msdn.microsoft.com/en-us/library/system.xml.linq.xdocument%28v=vs.110%29.aspx

您当然还有其他选择:

• SQL Server具有XML功能(请查看OPENXML)
• SSIS:您在这里提到了有关内存使用的问题，但这是一个选择.
• XSLT:在这种情况下，可能不如使用XmlReader那样好，但是您可能能够创建XSLT，然后再从XML创建SQL查询.

• SQL Server has XML functionality (look into OPENXML)
• SSIS: you mention concerns about memory usage here, but it's an option.
• XSLT: probably not as good an option as using XmlReader in this case, but you might be able to create XSLT that would then create a SQL query from your XML.

这篇关于有效地解析大型XML的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！