LevelDB与std :: map [英] LevelDB vs. std::map

问题描述

在我们的应用程序中，我们使用 std :: map 存储（关键，值）数据，并使用序列化将数据存储在磁盘上。使用这种方法，我们发现磁盘I / O是性能瓶颈，使用键查找值不是很快。

In our application we use std::map to store (key, value) data and use serialization to store that data on disk. With this approach we are finding that the disk I/O is performance bottleneck and finding values using key is not very fast.

我遇到过LevelDB并想使用它。但我有一些问题。

I have come across LevelDB and thinking of using it. But I have some questions.

1. LevelDB的文档说明它为（string，string）键值对。是否意味着我不能使用自定义键值对？


2. 似乎 std :: map 和LevelDB之间的区别是LevelDB是永久性的， std :: map 在内存中工作。因此，这意味着磁盘I / O瓶颈对于levelDB将更有问题。

1. LevelDB's documentation says its made for (string, string) key value pair. Does it mean that I can not use for custom key value pairs?
2. It seems the difference between std::map and LevelDB is that LevelDB is persistent and std::map works in memory. So does it mean the disk I/O bottleneck will be more problematic for levelDB.

更具体地说，任何人都可以解释一下LevelDB是否比 std :: map ？

More specifically can anybody please explain if LevelDB could be better choice than std::map?

PS：我尝试使用 hash_map s，但似乎慢于 std： ：map

PS: I tried using hash_maps but it appears to be slower than std::map

推荐答案

LevelDB只是比std :: map更容易。

LevelDB just does something else than std::map.

你真的说你想要std :: map的（高性能）持久性吗？

Are you really saying you want (high performance) persistence for std::map?

• p>看看std :: map与自定义分配器。从内存映射区域分配条目，并使用fsync来确保信息在战略性时刻击中磁盘。

• look at std::map with a custom allocator. Allocate the entries from a memory mapped region and use fsync to to ensure the information hits the disk at strategic moments in time.

• mmap


• boost iostreams内存映射文件

• mmap
• boost iostreams memory mapped files

也许结合EASTL（它拥有一个更快的std :: map和繁荣与自定义分配器 - 事实上他们没有默认分配器）

perhaps combine that with EASTL (which boasts a faster std::map and thrives with custom allocators - in fact they have no default allocator)

• EASTL

• EASTL

看看调整你的hash_map（std :: unorderded_map）;如果hash_maps较慢，你应该看看（a）loadfactor（b）哈希函数调整

look at tuning your hash_map (std::unorderded_map); if hash_maps are slower, you should look into (a) loadfactor (b) hash function tuning

• docs

• docs

最后最重要的：评估使用Boost序列化的二进制序列化你的地图（无论你选择什么实现）。根据我的经验，Boost序列化效果是帐单的顶部。

last but not least: evaluate the use of Boost Serialization for binary serialization of your map (whatever implementation you picked). In my experience Boost Serialization performance is top of the bill.

• 重复序列化

• Boost serialization

这篇关于LevelDB与std :: map的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！