使用Json.NET序列化为NDJSON [英] Serialize as NDJSON using Json.NET

问题描述

是否可以使用Json.NET序列化为 NDJSON (换行分隔的JSON)? Elasticsearch API使用NDJSON进行批量操作，我发现没有任何迹象表明任何 .NET库都支持这种格式.

Is it possible to serialize to NDJSON (Newline Delimited JSON) using Json.NET? The Elasticsearch API uses NDJSON for bulk operations, and I can find nothing suggesting that this format is supported by any .NET libraries.

此答案为反序列化NDJSON提供了指导，指出可以独立地对每一行进行序列化并与换行符连接，但我不一定称其为 supported .

This answer provides guidance for deserializing NDJSON, and it was noted that one could serialize each row independently and join with newline, but I would not necessarily call that supported.

推荐答案

由于Json.NET当前没有将集合序列化为NDJSON的内置方法，所以最简单的答案是将其写入单个TextWriter为每行使用单独的 JsonTextWriter ，设置 CloseOutput = false :

As Json.NET does not currently have a built-in method to serialize a collection to NDJSON, the simplest answer would be to write to a single TextWriter using a separate JsonTextWriter for each line, setting CloseOutput = false for each:

public static partial class JsonExtensions
{
    public static void ToNewlineDelimitedJson<T>(Stream stream, IEnumerable<T> items)
    {
        // Let caller dispose the underlying stream 
        using (var textWriter = new StreamWriter(stream, new UTF8Encoding(false, true), 1024, true))
        {
            ToNewlineDelimitedJson(textWriter, items);
        }
    }

    public static void ToNewlineDelimitedJson<T>(TextWriter textWriter, IEnumerable<T> items)
    {
        var serializer = JsonSerializer.CreateDefault();

        foreach (var item in items)
        {
            // Formatting.None is the default; I set it here for clarity.
            using (var writer = new JsonTextWriter(textWriter) { Formatting = Formatting.None, CloseOutput = false })
            {
                serializer.Serialize(writer, item);
            }
            // https://web.archive.org/web/20180513150745/http://specs.okfnlabs.org/ndjson/
            // Each JSON text MUST conform to the [RFC7159] standard and MUST be written to the stream followed by the newline character \n (0x0A). 
            // The newline charater MAY be preceeded by a carriage return \r (0x0D). The JSON texts MUST NOT contain newlines or carriage returns.
            textWriter.Write("\n");
        }
    }
}

示例小提琴.

由于单个NDJSON行可能很短，但行数可能很大，因此此答案提出了一种流传输解决方案，以避免分配大于85kb的单个字符串的必要性.如 Newtonsoft Json.NET性能提示所述，这样的大字符串最终会出现在

Since the individual NDJSON lines are likely to be short but the number of lines might be large, this answer suggests a streaming solution to avoid the necessity of allocating a single string larger than 85kb. As explained in Newtonsoft Json.NET Performance Tips, such large strings end up on the large object heap and may subsequently degrade application performance.

这篇关于使用Json.NET序列化为NDJSON的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！