我应该为一个非常大的数据集使用`HashSet`或`TreeSet`吗？ [英] Should I use a `HashSet` or a `TreeSet` for a very large dataset?

问题描述

我需要在数据结构中存储2到1,500万个帐户（长度为15的 String ），以便查找和检查唯一性。最初我计划将它们存储在 HashSet 中，但我怀疑由于哈希冲突导致查找的速度会很慢，并且最终会比TreeMap慢（使用二进制搜索）。

I have a requirement to store 2 to 15 million Accounts (which are a String of length 15) in a data structure for lookup purpose and checking uniqueness. Initially I planned to store them in a HashSet, but I doubt the speed of the lookup will be slow because of hash collisions and will eventually be slower than a TreeMap (using Binary search).

不需要对数据进行排序。我正在使用Java 7.我有64G系统，48G专用于此应用程序。

There is no requirement for Data to be sorted. I am using Java 7. I have 64G system with 48G dedicated for this application.

此问题不是 HashSet和TreeSet性能测试，因为该问题是关于将元素添加到 Set 这个问题是关于检查现有 Set 的重复值的表现。

This question is not a duplicate of HashSet and TreeSet performance test because that question is about the performance of adding elements to a Set and this question is about the performance of checking an existing Set for duplicate values.

推荐答案

当我们尝试使用适当的初始化参数在HashMap中存储5000万条记录时，插入开始减速，特别是在3500万条记录之后。更改为TreeMap提供了恒定的插入和检索性能。

When we tried to store 50 million records in HashMap with proper initialization parameters, insertion started to slowdown, especially after 35 million records. Changing to TreeMap gave a constant insertion and retrieval performance.

观察：对于大型输入集，TreeMap将提供比HashMap更好的性能。对于较小的集合，HashMap当然会提供更好的性能。

Observation : TreeMap will give better performance than a HashMap for large input set. For a smaller set, of course HashMap will give better performance.

这篇关于我应该为一个非常大的数据集使用`HashSet`或`TreeSet`吗？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！