使用 Json.NET 序列化为 NDJSON [英] Serialize as NDJSON using Json.NET

问题描述

是否可以使用 Json.NET 序列化为 NDJSON(换行符分隔的 JSON)?Elasticsearch API 使用 NDJSON 进行批量操作，我找不到任何表明 任何 .NET 库支持这种格式的信息.

Is it possible to serialize to NDJSON (Newline Delimited JSON) using Json.NET? The Elasticsearch API uses NDJSON for bulk operations, and I can find nothing suggesting that this format is supported by any .NET libraries.

这个答案提供了反序列化 NDJSON 的指导，它是注意到可以独立序列化每一行并使用换行符加入，但我不一定称其为支持.

This answer provides guidance for deserializing NDJSON, and it was noted that one could serialize each row independently and join with newline, but I would not necessarily call that supported.

推荐答案

由于 Json.NET 目前没有将集合序列化为 NDJSON 的内置方法，因此最简单的答案是写入单个 TextWriter 使用单独的 JsonTextWriter 为每一行设置 CloseOutput = false 对于每个:

As Json.NET does not currently have a built-in method to serialize a collection to NDJSON, the simplest answer would be to write to a single TextWriter using a separate JsonTextWriter for each line, setting CloseOutput = false for each:

public static partial class JsonExtensions
{
    public static void ToNewlineDelimitedJson<T>(Stream stream, IEnumerable<T> items)
    {
        // Let caller dispose the underlying stream 
        using (var textWriter = new StreamWriter(stream, new UTF8Encoding(false, true), 1024, true))
        {
            ToNewlineDelimitedJson(textWriter, items);
        }
    }

    public static void ToNewlineDelimitedJson<T>(TextWriter textWriter, IEnumerable<T> items)
    {
        var serializer = JsonSerializer.CreateDefault();

        foreach (var item in items)
        {
            // Formatting.None is the default; I set it here for clarity.
            using (var writer = new JsonTextWriter(textWriter) { Formatting = Formatting.None, CloseOutput = false })
            {
                serializer.Serialize(writer, item);
            }
            // https://web.archive.org/web/20180513150745/http://specs.okfnlabs.org/ndjson/
            // Each JSON text MUST conform to the [RFC7159] standard and MUST be written to the stream followed by the newline character 
 (0x0A). 
            // The newline charater MAY be preceeded by a carriage return 
 (0x0D). The JSON texts MUST NOT contain newlines or carriage returns.
            textWriter.Write("
");
        }
    }
}

示例 小提琴.

由于单个 NDJSON 行可能很短但行数可能很大，因此此答案建议采用流式解决方案来避免分配大于 85kb 的单个字符串的必要性.如 Newtonsoft Json.NET 性能提示中所述，如此大的字符串最终会出现在 大型对象堆 并可能随后降低应用程序性能.

Since the individual NDJSON lines are likely to be short but the number of lines might be large, this answer suggests a streaming solution to avoid the necessity of allocating a single string larger than 85kb. As explained in Newtonsoft Json.NET Performance Tips, such large strings end up on the large object heap and may subsequently degrade application performance.

这篇关于使用 Json.NET 序列化为 NDJSON的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！