处理巨大的HTTP JSON响应的最佳方法 [英] Best way to process a huge HTTP JSON response

问题描述

从返回为1条大行的服务器解析JSON回复的最有效方法是什么?我不需要/不需要将此JSON映射"到某些自定义/业务对象，因为我只需要结构的特定部分.
主要在一组数组中查找特定标签及其值.

What is the most efficient way to parse a JSON reply from a server that is return as 1 huge line? I don’t want/need to "map" this JSON to some custom/business objects as I need only specific parts of the structure.
Mainly finding a specific tag and its values among a set of arrays.

更新:

我正在寻找一种有效的方法来解析来自内部服务器的JSON响应.
响应以单行形式发送.该文件太大，以至于无法使用浏览器的json插件来了解其结构，因为浏览器被卡住"了.
对于我的需求，我需要特定的信息位，并且不想将JSON映射到实际的业务对象/类，因为这将是不必要的工作.我目前为此做些什么(为了清楚起见，我已删除了异常处理):

I am looking for an efficient way to parse an JSON response from an internal server.
The response is sent as a huge single line. The file is so big that trying to understand the structure using brower’s json plugin was still difficult as the browser was "stuck".
For my needs I want specific bits of information and do not want to map the JSON to actual business object/classes as it will be too much unneeded work. What I currently do for testing this out (I have removed exception handling for clarity):

HttpClient client = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(url);
HttpResponse response = null;
response = client.execute(httpGet);
StatusLine statusLine = response.getStatusLine();
int statusCode = statusLine.getStatusCode();
StringBuilder builder = null;
if (statusCode == 200) {
          HttpEntity entity = response.getEntity();
          InputStream content = entity.getContent();
          builder = new StringBuilder();
          BufferedReader reader = new BufferedReader(new InputStreamReader(content));
          String line;
          while ((line = reader.readLine()) != null) {
                    builder.append(line);
          }
          if ( builder.length() != 0 ) {
                processJSON(builder.toString());
          }
           return builder.toString();
}

private void processJSON(String s) {
        JSONObject o = new JSONObject(s);
        String url = (String) o.get("url");

       JSONArray array = o.getJSONArray("individualroles");

       for( int i = 0; i < array.length(); i++ ) {
                JSONObject elem = (JSONObject) array.get(i);
                String fullNameForDisplay = (String) elem.get("fullName");
                String status =  (String) elem.get("status");
                String date = (String) elem.get("date");
                String fullDescription = (String) elem.get("full_description").toString();
                JSONArray contacts = (JSONArray) elem.get("contacts");
                for(int j = 0; j < contacts.length(); j ++ ) {
                    JSONObject contact = (JSONObject) contacts.get(j);
                    String contactName = contact.getString("contactName");
                    String displayName = contact.getString("displayName");
                    String realAddress = contact.getString("realAddress");

                    Log.i("MyApp", "contactName = " + contactName + " displayName = " + displayName + " readAddress  = " + realAddress);
                }
         }

这实际上是作为JSON响应的日志，即 String s 的长度约为700万个字符.
我还在日志中看到许多语句，例如:

This literally fluds the log as the JSON response i.e. the String s has length of ~7Million characters.
Also I see in the log many statements like:

I/art:背景粘性并发标记清除GC已释放389(14KB)个AllocSpace对象，3个(851KB)LOS对象，0％空闲，2MB/2MB，已暂停30.651ms，总计77.127ms

I/art﹕ Background sticky concurrent mark sweep GC freed 389(14KB) AllocSpace objects, 3(851KB) LOS objects, 0% free, 2MB/2MB, paused 30.651ms total 77.127ms

要对此进行测试，我要做的是在片段的 onCreate 中执行

For testing this what I did is that in the fragment’s onCreate I do:

 Thread thread = new Thread(new Runnable() {
            @Override
            public void run() {
                Log.i("MyApp", (new JSONParser().get(JSON_URL_SERVER)));
               }
        });
        thread.start();

是否存在任何问题，例如这种方法可以处理内存和GC?我该如何改善呢?

Is there any issue e.g. with memory and GC with this approach? How can I improve this?

推荐答案

您应该使用一些在读取过程中解析输入流的东西.

You should use something that parses the inputstream during reading.

类似于XML的SAX解析器，JSON也有此类实现.杰克逊(Jackson)可以同时使用 json-simple ，它的处理方法非常简单，此处解释.

Similar to SAX parsers for XML there are such implementations for JSON too. There's Jackson for one and alsp json-simple which has a quite simple approach to it, expained here.

如果我正确理解您的代码，那么您主要是在键"individualroles" 下查找所有内容.因此，使用此类似于SAX的解析器，您将在示例中实现 ContentFinder (如 KeyFinder )，并在解析器中一旦到达所需的键，就会调用该解析器.输入流.现在，您可以处理JSON的那部分，然后可以一起结束解析/读取.

If I understand your code right, you are mainly looking for everything under key "individualroles". So with this SAX-like parser you'd implement a ContentFinder like the KeyFinder in the example and that is invoked by the Parser as soon as the required key is reached in the inputstream. Now you can handle that part of the JSON and then you can end the parsing/reading alltogether.

很抱歉，我无法提供更详细的说明，但我自己并未使用该特定库，这只是我可以提供的映射到JSON的SAX解析器中的知识.

I'm sorry I cannot provide a more detailed explanation but I haven't used that particular library myself, it's just knowledge from SAX-parsers mapped to JSON I can provide.

这篇关于处理巨大的HTTP JSON响应的最佳方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！