我应该为一个非常大的数据集使用`HashSet`或`TreeSet`吗? [英] Should I use a `HashSet` or a `TreeSet` for a very large dataset?
问题描述
我需要在数据结构中存储2到1,500万个帐户(长度为15的 String
),以便查找和检查唯一性。最初我计划将它们存储在 HashSet
中,但我怀疑由于哈希冲突导致查找的速度会很慢,并且最终会比TreeMap慢(使用二进制搜索)。
I have a requirement to store 2 to 15 million Accounts (which are a String
of length 15) in a data structure for lookup purpose and checking uniqueness. Initially I planned to store them in a HashSet
, but I doubt the speed of the lookup will be slow because of hash collisions and will eventually be slower than a TreeMap (using Binary search).
不需要对数据进行排序。我正在使用Java 7.我有64G系统,48G专用于此应用程序。
There is no requirement for Data to be sorted. I am using Java 7. I have 64G system with 48G dedicated for this application.
此问题不是 HashSet和TreeSet性能测试,因为该问题是关于将元素添加到 Set
这个问题是关于检查现有 Set
的重复值的表现。
This question is not a duplicate of HashSet and TreeSet performance test because that question is about the performance of adding elements to a Set
and this question is about the performance of checking an existing Set
for duplicate values.
推荐答案
当我们尝试使用适当的初始化参数在HashMap中存储5000万条记录时,插入开始减速,特别是在3500万条记录之后。更改为TreeMap提供了恒定的插入和检索性能。
When we tried to store 50 million records in HashMap with proper initialization parameters, insertion started to slowdown, especially after 35 million records. Changing to TreeMap gave a constant insertion and retrieval performance.
观察:对于大型输入集,TreeMap将提供比HashMap更好的性能。对于较小的集合,HashMap当然会提供更好的性能。
Observation : TreeMap will give better performance than a HashMap for large input set. For a smaller set, of course HashMap will give better performance.
这篇关于我应该为一个非常大的数据集使用`HashSet`或`TreeSet`吗?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!