使用 Json.net 将大量数据流式传输为 JSON 格式 [英] Streaming large list of data as JSON format using Json.net

问题描述

使用 MVC 模型，我想编写一个 JsonResult，它将 Json 字符串流式传输到客户端，而不是将所有数据一次转换为 Json 字符串，然后将其流式传输回客户端.我有一些动作需要在 Json 传输时发送非常大(超过 300,000 条记录)，我认为基本的 JsonResult 实现是不可扩展的.

Using the MVC model, I would like to write a JsonResult that would stream the Json string to the client rather than converting all the data into Json string at once and then streaming it back to the client. I have actions that require to send very large (over 300,000 records) as Json transfers and I think the basic JsonResult implementation is not scalable.

我正在使用 Json.net，我想知道是否有一种方法可以在转换 Json 字符串时对其进行流式传输.

I am using Json.net, I am wondering if there is a way to stream the chunks of the Json string as it is being transformed.

//Current implementation:
response.Write(Newtonsoft.Json.JsonConvert.SerializeObject(Data, formatting));
response.End();

//I know I can use the JsonSerializer instead
Newtonsoft.Json.JsonSerializer serializer = new Newtonsoft.Json.JsonSerializer();
serializer.Serialize(textWriter, Data);

但是我不确定如何将块写入 textWriter 并写入响应并调用 reponse.Flush() 直到所有 300,000 条记录都转换为 Json.

However I am not sure how I can get the chunks written into textWriter and write into response and call reponse.Flush() until all 300,000 records are converted to Json.

这可能吗?

推荐答案

假设您的最终输出是一个 JSON 数组，并且每个块"是该数组中的一项，您可以尝试类似以下 JsonStreamingResult类.它使用 JsonTextWriter 将 JSON 写入输出流，并使用 JObject 作为在将每个项目写入写入器之前单独序列化每个项目的方法.您可以向 JsonStreamingResult 传递一个 IEnumerable 实现，该实现可以从您的数据源中单独读取项目，这样您就不会一次将它们全部放在内存中.我没有对此进行过广泛的测试，但它应该能让你朝着正确的方向前进.

Assuming your final output is a JSON array and each "chunk" is one item in that array, you could try something like the following JsonStreamingResult class. It uses a JsonTextWriter to write the JSON to the output stream, and uses a JObject as a means to serialize each item individually before writing it to the writer. You could pass the JsonStreamingResult an IEnumerable implementation which can read items individually from your data source so that you don't have them all in memory at once. I haven't tested this extensively, but it should get you going in the right direction.

public class JsonStreamingResult : ActionResult
{
    private IEnumerable itemsToSerialize;

    public JsonStreamingResult(IEnumerable itemsToSerialize)
    {
        this.itemsToSerialize = itemsToSerialize;
    }

    public override void ExecuteResult(ControllerContext context)
    {
        var response = context.HttpContext.Response;
        response.ContentType = "application/json";
        response.ContentEncoding = Encoding.UTF8;

        JsonSerializer serializer = new JsonSerializer();

        using (StreamWriter sw = new StreamWriter(response.OutputStream))
        using (JsonTextWriter writer = new JsonTextWriter(sw))
        {
            writer.WriteStartArray();
            foreach (object item in itemsToSerialize)
            {
                JObject obj = JObject.FromObject(item, serializer);
                obj.WriteTo(writer);
                writer.Flush();
            }
            writer.WriteEndArray();
        }
    }
}

这篇关于使用 Json.net 将大量数据流式传输为 JSON 格式的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！