类型提供程序和大XML文件 [英] Type provider and big XML file

问题描述

我想知道在F#中使用类型提供程序有哪些好的做法，

I am wondering what are the good practices in using type provider in F#,

我有一个XML文件(25Mo +)，我认为这不是问题，但是我的Visual Studio遇到了很多麻烦.通常的最佳做法是用最少的数据定义XML的完整模板，然后再加载内容?

I have an XML file (25Mo+), I thought it will not be an issue but my Visual Studio is suffering a lot loll. What is usually the good practice, to define a complete template of the XML with the minimum of data and Load the content later ?

如果我们加载一个带有可选节点的大文件，那么类型将由最完整的文件推断出来?

And if we load a big file with optional node, the type will be inferred by the most complete one ?

谢谢

推荐答案

XmlProvider类型提供程序基于XDocument(LINQ to XML)，因此始终需要将文件读入内存.但是，可能会由于模式推断而导致一些额外的开销，您可以避免这些开销.因此，如果LINQ to XML可以读取25MB的文件(我不确定，请尝试在文件上使用XDocument.Load)，则可以使用类型提供程序.

The XmlProvider type provider is based on XDocument (LINQ to XML), so it always needs to read the file into memory. However, there may be some additional overhead caused by the schema inference, which you can avoid. So, if LINQ to XML can read 25MB file (I'm not sure, try using XDocument.Load on the file), then you would be able to use the type provider.

假设big.xml是您的大文件，您可以尝试从文件中删除某些元素(以便所有结构都存在，但是更小)并创建small.xml.然后您可以使用:

Assuming big.xml is your big file, you can try deleting some of the elements from the file (so that all structure is there, but it is smaller) and creating small.xml. Then you can use:

type X = XmlProvider<"small.xml">
let data = X.Load("big.xml")

这只会在较小的文件(在Visual Studio中在后台运行)上运行模式推断，然后尝试使用LINQ to XML读取较大的文件.这只是读取文件，因此，如果LINQ to XML可以读取25MB的文件，则可以使用.

This will only run schema inference on the smaller file (which runs in background in Visual Studio) and then it attempts to read the bigger file using LINQ to XML. This just reads the file, so if LINQ to XML can read 25MB file, this will work.

理想情况下，我们将对可以在XmlReader上运行的类型提供程序进行一些更改或类似的操作-可以重用某些F#数据基础结构，但仍然需要大量工作(打开问题是您感兴趣的贡献！)

Ideally, we would have some variation on the type provider that works over XmlReader or something like that - this could reuse some of F# Data infrastructure, but it would still be a lot of work (open an issue at F# Data is you're interested in contributing!)

这篇关于类型提供程序和大XML文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！