使用太多内存的嵌套STL向量 [英] Nested STL vector using way too much memory

问题描述

我有一个Partition个对象的STL向量My_Partition_Vector，定义为

I have an STL vector My_Partition_Vector of Partition objects, defined as

struct Partition // the event log data structure
{
    int key;
    std::vector<std::vector<char> > partitions;
    float modularity;
};

Partition.partitions的实际嵌套结构因对象而异，但分区中存储的字符总数为16.分区始终为16.

The actual nested structure of Partition.partitions varies from object to object but in the total number of chars stored in Partition.partitions is always 16.

因此，我假设对象的总大小应或多或少为24个字节(16 + 4 + 4).但是，我添加到My_Partition_Vector的每100,000个项目，内存消耗(使用ps -aux查找)将增加大约20 MB，指示每个分区对象大约209个字节.

I assumed therefore that the total size of the object should be more or less 24 bytes (16 + 4 + 4). However for every 100,000 items I add to My_Partition_Vector, memory consumption (found using ps -aux) increases by around 20 MB indicating around 209 bytes for each Partition Object.

这是近9倍的增加！?这些额外的内存使用量是从哪里来的? STL向量或结构中的某种填充?我该如何解决(并阻止它进入交换)?

This is a nearly 9 Fold increase!? Where is all this extra memory usage coming from? Some kind of padding in the STL vector, or the struct? How can I resolve this (and stop it reaching into swap)?

推荐答案

一件事，std::vector为动态数组建模，因此如果您知道使用std::vector在partitions中总是有16个字符，那就太过分了.使用良好的旧C样式数组/矩阵， boost :: array 或 boost :: multi_array .

For one thing std::vector models a dynamic array so if you know that you'll always have 16 chars in partitions using std::vector is overkill. Use a good old C style array/matrix, boost::array or boost::multi_array.

由于内存布局限制，为了减少插入/添加元素所需的重新分配次数，允许std::vector预先为一定数量的元素预分配内存(并且capacity()成员函数将告诉您如何很多).

To reduce the number of re-allocations needed for inserting/adding elements due to it's memory layout constrains std::vector is allowed to preallocate memory for a certain number of elements upfront (and it's capacity() member function will tell you how much).

这篇关于使用太多内存的嵌套STL向量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！