序列化 XML 以使用 Java 流式传输的最佳方法? [英] Best approach to serialize XML to stream with Java?

问题描述

我们使用 XStream 对 XML 进行序列化/反序列化...但刚刚收到一个 OutOfMemory 异常.

We serialize/deserialize XML using XStream... and just got an OutOfMemory exception.

首先我不明白为什么我们会收到错误，因为我们有 500MB 分配给服务器.

Firstly I don't understand why we're getting the error as we have 500MB allocated to the server.

问题是 - 我们应该做出哪些改变才能避免麻烦?我们希望确保此实施的规模.

Question is - what changes should we make to stay out of trouble? We want to ensure this implementation scales.

目前我们有 ~60K 个对象，每个 ~50 字节.我们在内存中加载 60K POJO，并将它们序列化为一个字符串，我们使用 HttpClient 将其发送到 Web 服务.接收时，我们获取整个String，然后转换为POJO.XML/对象层次结构类似于:

Currently we have ~60K objects, each ~50 bytes. We load the 60K POJO's in memory, and serialize them to a String which we send to a web service using HttpClient. When receiving, we get the entire String, then convert to POJO's. The XML/object hierarchy is like:

<root>
    <meta>
       <date>10/10/2009</date>
       <type>abc</type>
    </meta>

    <data>
        <field>x</field>
    </data>

    [thousands of <data>]
</root>

我认为最好的方法是不要将 POJO 存储在内存中，并且不要将内容写入单个字符串.相反，我们应该将各个 POJO 写入流.XStream 支持，但似乎 元素不会不支持.数据需要采用以下形式:

I gather the best approach is to not store the POJO's in memory and not write the contents to a single String. Instead we should write the individual <data> POJO's to a stream. XStream supports this but seems like the <meta> element wouldn't be supported. Data would need to be in form:

<root> 
    <data>
        <field>x</field>
    </data>

    [thousands of <data>]
</root>

那么什么方法最容易流式传输整个树?

So what approach is easiest to stream the entire tree?

推荐答案

您肯定希望避免将 POJO 序列化为一个庞大的字符串，然后将该字符串写出.使用 XStream API 将 POJO 直接序列化到您的 OutputStream.今年早些时候我遇到了同样的情况，当时我发现我正在生成 200-300Mb 的 XML 文档并得到 OutOfMemoryErrors.进行切换非常容易.

You definitely want to avoid serializing your POJOs into a humongous String and then writing that String out. Use the XStream APIs to serialize the POJOs directly to your OutputStream. I ran into the same situation earlier this year when I found that I was generating 200-300Mb XML documents and getting OutOfMemoryErrors. It was very easy to make the switch.

当然，阅读方面也是如此.不要将 XML 读入字符串并要求 XStream 从该字符串反序列化:直接从 InputStream 反序列化.

And ditto of course for the reading side. Don't read the XML into a String and ask XStream to deserialize from that String: deserialize directly from the InputStream.

您提到了关于无法序列化 元素和 元素的第二个问题.我不认为这是 XStream 问题或限制，因为我经常按以下顺序序列化更复杂的结构:

You mention a second issue regarding not being able to serialize the <meta> element and the <data> elements. I don't think this is an XStream problem or limitation as I routinely serialize much more complex structures on the order of:

<myobject>
    <item>foo</item>
    <anotheritem>foo</anotheritem>
    <alist>
        <alistitem>
            <value1>v1</value1>
            <value2>v2</value2>
            <value3>v3</value3>
            ...
        </alistitem>
        ...
        <alistitem>
            <value1>v1</value1>
            <value2>v2</value2>
            <value3>v3</value3>
            ...
        </alistitem>
    </alist>
    <anotherlist>
        <anotherlistitem>
            <valA>A</valA>
            <valB>B</valB>
            <valC>C</valC>
            ...
        </anotherlistitem>
        ...
    </anotherlist>
</myobject>

我也成功地序列化和反序列化了嵌套列表.

I've successfully serialized and deserialized nested lists too.

这篇关于序列化 XML 以使用 Java 流式传输的最佳方法?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！