快速与在Java中逐行读取大型JSON文件的有效方法 [英] Fast & Efficient Way To Read Large JSON Files Line By Line in Java

问题描述

我在 JSON 文件中有1亿条记录，需要一种高效，最快的方法来从 JSON java 中的c $ c>文件。

I have 100 millions of records in JSON file, need an efficient and fastest method to read the array of arrays from a JSON file in java.

JSON 文件格式如下：

[["XYZ",...,"ABC"],["XYZ",...,"ABC"],["XYZ",...,"ABC"],...,["XYZ",...,"ABC"],
 ["XYZ",...,"ABC"],["XYZ",...,"ABC"],["XYZ",...,"ABC"],...,["XYZ",...,"ABC"],
 ...
 ...
 ...
 ,["XYZ",...,"ABC"],["XYZ",...,"ABC"],["XYZ",...,"ABC"]]

我想逐行阅读此 JSON 文件：

先阅读：

["XYZ",...,"ABC"]

然后：

["XYZ",...,"ABC"]

等等：'

...
...
...
["XYZ",...,"ABC"]

如何读取这样的 JSON 文件，我知道它看起来并不完全ea JSON 文件，但我需要以另存为这种格式读取此文件。JSON

How do I read a JSON file like this, I know it does not completely look like a JSON file but I need to read this file in this format which is saved as.JSON

推荐答案

您可以使用 JSON处理API（JSR 353）处理您的数据以流式传输方式：

You can use JSON Processing API (JSR 353), to process your data in a streaming fashion:

import javax.json.Json;
import javax.json.stream.JsonParser;

...

String dataPath = "data.json";

try(JsonParser parser = Json.createParser(new FileReader(dataPath))) {
     List<String> row = new ArrayList<>();

     while(parser.hasNext()) {
         JsonParser.Event event = parser.next();
         switch(event) {
             case START_ARRAY:
                 continue;
             case VALUE_STRING:
                 row.add(parser.getString());
                 break;
             case END_ARRAY:
                 if(!row.isEmpty()) {
                     //Do something with the current row of data 
                     System.out.println(row);

                     //Reset it (prepare for the new row) 
                     row.clear();
                 }
                 break;
             default:
                 throw new IllegalStateException("Unexpected JSON event: " + event);
         }
     }
}

这篇关于快速与在Java中逐行读取大型JSON文件的有效方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！