在后台优化解析大型JSON [英] Optimal parsing of large JSON in background

问题描述

我正在解析类似于类似这样的大型JSON数据.

I am parsing large JSON data similar to one like this.

这里大约有3000多个chapter对象.我只需要带有"lang_code":"gb"的那些chapter对象，这些对象大约是1300，其中包含一些基本对象，例如title，description.因此，基本上55％的JSON不适合我使用.

There are around 3000+ chapter objects here. I require only those chapter objects with "lang_code":"gb", which will be around 1300 with some basic objects like title, description. So basically 55% of JSON is not for my use.

我正在使用 https://app.quicktype.io/生成用于JSON解析的类. 这给了我正确的类，但是这种方法太慢了.

I am generating the classes for JSON parsing using https://app.quicktype.io/ which gives me correct classes but this method is too slow.

任何加快速度的建议.

推荐答案

如果您所需的大多数信息都是稀疏的，则最好以针对性的方式挑选出来，而不是为所有内容创建对象.

If most of the information that you need is sparse, it's probably best to pick it out in a targeted fashion, rather than create objects for everything.

您无法解码整个json字符串，这在我的笔记本电脑上大约需要60毫秒.修剪非gb章节仅需几毫秒，而将剩余内容映射到某些可用对象还需要几毫秒.可用的总时间:小于70毫秒.

You can't get around decoding the whole json string, which takes about 60ms on my laptop. Pruning the non-gb chapters takes just a few ms, and mapping what's left to some usable objects takes another few ms. Total time to something usable: <70ms.

import 'dart:convert';
import 'dart:io';

main() {
  String manga = new File('manga.json').readAsStringSync();
  int t1 = DateTime.now().millisecondsSinceEpoch;

  Map<String, dynamic> data = json.decode(manga);
  Map<String, dynamic> jChapters = data['chapter'];
  jChapters.removeWhere((_, m) => m['lang_code'] != 'gb');

  Map<String, Chapter> chapters = jChapters.map((_, m) {
    String number = m['chapter'];
    return MapEntry(number, Chapter(number, m['title']));
  });

  int t2 = DateTime.now().millisecondsSinceEpoch;
  print(t2 - t1);

  print(chapters);
}

class Chapter {
  String number;
  String title;

  Chapter(this.number, this.title);

  @override
  String toString() => 'Chapter #$number:$title';
}

这篇关于在后台优化解析大型JSON的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！