为什么Java HashMap变慢？ [英] Why is Java HashMap slowing down?

问题描述

我试图用一个文件的内容来构建一个地图，我的代码如下：

  System.out。 println（开始建立sns地图....）; 
 String basePath = PropertyReader.getProp（oldbasepath）; 
 String pathname = basePath +\\user_sns.txt; 
 FileReader fr; 
 Map< Integer，List< Integer>> snsMap = 
 new HashMap< Integer，List< Integer>>（2000000）; 
尝试{
 fr = new FileReader（pathname）; 
 BufferedReader br = new BufferedReader（fr）; 
字符串行; 
 int i = 1; （（line = br.readLine（））！= null）{
 System.out.println（line number：+ i）; 
 i ++; 
 
 String [] strs = line.split（\t）; 
 int key = Integer.parseInt（strs [0]）; 
 int value = Integer.parseInt（strs [1]）; 
列表<整数> list = snsMap.get（key）; 
 //如果追随者不在地图中
 if（snsMap.get（key）== null）
 list = new LinkedList< Integer>（）; 
 list.add（value）; 
 snsMap.put（key，list）; 
 System.out.println（map size：+ snsMap.size（））; 
} 
} catch（IOException e）{
 e.printStackTrace（）; 
} 
 System.out.println（完成建立sns地图....）; 
返回snsMap; 
  

程序在开始时速度非常快<，但信息缓慢打印为：

 地图大小：1138338 
行号：30923602 
地图大小：1138338 
行号：30923603 
 .... 
  

我尝试找到两个理由System.out.println（）子句用于判断BufferedReader和HashMap的性能，而不是Java Profiler。
在获取行号信息后，有时需要一段时间才能获取地图大小的信息，有时，获取地图大小后需要一段时间才能获取行号信息。我的问题是：这会让我的程序变慢吗？大型文件的BufferedReader或大型地图的HashMap？ 如果您是在Eclipse内部测试此项功能，那么您应该了解巨大性能损失由于Eclipse在控制台视图中捕获了输出，所以写入stdout / stderr。但是，如果你抱怨的是处理3000万行之后经历的放缓，那么在严格循环内打印始终是一个性能问题，即使在Eclipse之外也是如此。 我敢打赌这是一个记忆问题。首先它由于激烈的GC'ing而变慢，然后它与 OutOfMemoryError 打破。

I try to build a map with the content of a file and my code is as below:

    System.out.println("begin to build the sns map....");
    String basePath = PropertyReader.getProp("oldbasepath");
    String pathname = basePath + "\\user_sns.txt";
    FileReader fr;
    Map<Integer, List<Integer>> snsMap = 
            new HashMap<Integer, List<Integer>>(2000000);
    try {
        fr = new FileReader(pathname);
        BufferedReader br = new BufferedReader(fr);
        String line; 
        int i = 1;
        while ((line = br.readLine()) != null) {
            System.out.println("line number: " + i);
            i++;

            String[] strs = line.split("\t");
            int key = Integer.parseInt(strs[0]);
            int value = Integer.parseInt(strs[1]);
            List<Integer> list = snsMap.get(key);
            //if the follower is not in the map
            if(snsMap.get(key) == null) 
                list = new LinkedList<Integer>();
            list.add(value);
            snsMap.put(key, list);
            System.out.println("map size: " + snsMap.size());
        }
    } catch (IOException e) {
        e.printStackTrace();
    }
    System.out.println("finish building the sns map....");
    return snsMap;

The program is very fast at first but gets much slowly when the information printed is :

 map size: 1138338
 line number: 30923602
 map size: 1138338
 line number: 30923603 
 ....

I try to find to reason with two System.out.println() clauses to judge the preformance of BufferedReader and HashMap instead of a Java profiler. Sometimes it takes a while to get the information of the map size after getting the line number information, and sometimes, it takes a while to get the information of the line number information after get the map size. My question is: which makes my program slow? the BufferedReader for a big file or HashMap for a big map?

解决方案

If you are testing this from inside Eclipse, you should be aware of the huge performance penalty of writing to stdout/stderr, due to Eclipse's capturing that ouptut in the Console view. Printing inside a tight loop is always a performance issue, even outside of Eclipse.

But, if what you are complaining about is the slowdown experienced after processing 30 million lines, then I bet it's a memory issue. First it slows down due to intense GC'ing and then it breaks with OutOfMemoryError.

这篇关于为什么Java HashMap变慢？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！