磁盘支持的STL容器类？ [英] Disk-backed STL container classes?

问题描述

我喜欢使用STL开发算法，但是，我的数据集对于堆来说太大了，我遇到了这个反复出现的问题。

我一直在寻找STL容器和磁盘备份算法（即存储在磁盘而不是堆上的数据结构）的插件替换。

朋友最近指示我 stxxl 。之前我太参与了它吗？有什么其他磁盘支持的STL替换可用，我应该考虑？

注意：我对持久性或嵌入式数据库不感兴趣。请不要提及boost :: serialization，POST ++，关系模板库，Berkeley DB，sqlite等。我知道这些项目，当它们适合我的目的使用它们。

更新：有几个人提到内存映射文件并使用自定义分配器，很好的建议BTW，但我会指出他们讨论这里，David Abraham建议对磁盘备份的容器需要定制迭代器。意思是自定义分配器方法是不可能的。

解决方案

我已经实现了一些非常相似的事情。实施迭代器是最具挑战性的。我使用了 boost :: iterator_facade 来实现迭代器。使用 boost :: iterator_facade ，您可以轻松适应任何缓存在磁盘数据结构上的STL容器接口。 / p>

I enjoy developing algorithms using the STL, however, I have this recurring problem where my data sets are too large for the heap.

I have been searching for drop-in replacements for STL containers and algorithms which are disk-backed, i.e. the data structures on stored on disk rather than the heap.

A friend recently pointed me towards stxxl. Before I get too involved with it... Are any other disk-backed STL replacements available that I should be considering?

NOTE: I'm not interested in persistence or embedded databases. Please don't mention boost::serialization, POST++, Relational Template Library, Berkeley DB, sqlite, etc. I am aware of these projects and use them when they are appropriate for my purposes.

UPDATE: Several people have mentioned memory-mapping files and using a custom allocator, good suggestions BTW, but I would point them to the discussion here where David Abraham suggests that custom iterators would be needed for disk-backed containers. Meaning the custom allocator approach isn't likely to work.

解决方案

I have implemented some thing very similar. Implementing the iterators is the most challenging. I used boost::iterator_facade to implement the iterators. Using boost::iterator_facade you can easy adapt any cached on disk data structures to have a STL container interface.

这篇关于磁盘支持的STL容器类？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！