stl向量中的内存映射文件存储 [英] Memory mapped file storage in stl vector

问题描述

我正在尝试实现自定义的 allocator ，用于将内存映射文件存储在 std :: vector 中.由 boost :: iostreams :: mapped_file

I'm trying to implement custom allocator for storing memory mapped files in the std::vector. Files mapping performed by boost::iostreams::mapped_file

用于文件内存映射的分配器类型:

Allocator type for file memory mapping:

template<typename T>
class mmap_allocator 
{
public:
  typedef T value_type;

  mmap_allocator(const std::string& filename) 
  : _mmfile(filename) {  } 

  T* allocate (size_t n) 
  { 
     return reinterpret_cast<T*>(_mmfile.data());
  }
  void deallocate (T* p, size_t n) 
  { 
     p = nullptr;
     _mmfile.close();
  }

private:
  boost::iostreams::mapped_file _mmfile;
};

用于内存映射文件的容器，基于 std :: vector :

Container for memory mapped file, based on std::vector:

//Get file size
long GetFileSize(std::string filename)
{
    FILE *p_file = NULL;
    p_file = fopen(filename.c_str(),"rb");
    fseek(p_file,0,SEEK_END);
    int size = ftell(p_file);
    fclose(p_file);
    return size;
}

template<typename T>
class mm_vector : public std::vector<T, mmap_allocator<T> >
{
public:
  typedef mmap_allocator<T> allocator_type;
  typedef std::vector<T, allocator_type > b_vector;

  mm_vector(const std::string filename) : b_vector(GetFileSize(filename)/sizeof(T), allocator_type(filename)) 
  {  
    b_vector::reserve(GetFileSize(filename)/sizeof(T));
  }
};

测试代码:

int main()
{
  mm_vector<int> v("test.f");//test.f - binary file contain several integers
  for(auto x : v) std::cout<<x<<"  ";
}

此代码无法正常工作-输出始终等于零.文件包含正确的内容-几个整数.这段代码效果很好:

This code don't work properly - output always equals to zero. File contains correct content - several integers. This code works well:

boost::iostreams::mapped_file _mmfile("test.f");
int* p = (int*)(_mmfile.data());
std::cout<<p[0];

我在做什么错了?

推荐答案

问题是零初始化，调用接收大小的构造函数，分配器会将矢量元素初始化为默认值元素类型(在这种情况下为0).这是标准规定的.

The problem is zero initialization, calling the constructor that receive the size and the allocator would initialize the vector elements to the default value of the element type (in this case 0). This is mandated by the standard.

23.3.7.2矢量构造函数，复制和分配[vector.cons] §23.3.7.2 789

23.3.7.2 vector constructors, copy, and assignment [vector.cons] § 23.3.7.2 789

explicit vector(size_type n, const Allocator& = Allocator());

-Effects:使用指定的分配器构造具有n个默认插入元素的向量.
-要求:T必须是DefaultInsertable到* this中.
-复杂度:线性n.

-Effects: Constructs a vector with n default-inserted elements using the specified allocator.
-Requires: T shall be DefaultInsertable into *this.
-Complexity: Linear in n.

在我的情况下，使用的文件也填充了0.在GCC 4.9.0中测试.具有逻辑性，因为被映射的文件的默认mapmode是 readwrite .

In my case the used file was filled with 0 too. Tested in GCC 4.9.0. Has logic because the default mapmode of mapped_file is readwrite.

在示例代码中，当发生分配(在自定义分配器中)，向量的构造并存在时，我添加了映射内存内容的打印主要打印.第一个 print 输出正确的文件数据，第二个输出零位版本.

In the sample code i added print of the mapped memory content when the allocation is happen (in the custom allocator), in the construction of the vector and the existed print in main. The first print output the correct data of the file and the second output the zeroed version.

#include <vector>
#include <iostream>
#include <chrono>
#include <iomanip>
#include <boost/iostreams/device/mapped_file.hpp>

template <typename T>
class mmap_allocator {
public:
    typedef T value_type;

    mmap_allocator(const std::string& filename) : _mmfile(filename) {}

    T* allocate(size_t n) {
        std::cout << "OUTPUT 1:" << std::endl;
        auto v = reinterpret_cast<T*>(_mmfile.data());
        for (unsigned long idx = 0; idx < _mmfile.size()/sizeof(int); idx++)
            std::cout << v[idx] << " ";
        return reinterpret_cast<T*>(_mmfile.data());
    }
    void deallocate(T* p, size_t n) {
        p = nullptr;
        _mmfile.close();
    }

private:
    boost::iostreams::mapped_file _mmfile;
};

// Get file size
long GetFileSize(std::string filename) {
    FILE* p_file = NULL;
    p_file = fopen(filename.c_str(), "rb");
    fseek(p_file, 0, SEEK_END);
    int size = ftell(p_file);
    fclose(p_file);
    return size;
}

template <typename T>
class mm_vector : public std::vector<T, mmap_allocator<T>> {
public:
    typedef mmap_allocator<T> allocator_type;
    typedef std::vector<T, allocator_type> b_vector;

    mm_vector(const std::string filename)
        : b_vector(GetFileSize(filename) / sizeof(T),
                   allocator_type(filename)) {
        std::cout << std::endl << std::endl << "OUTPUT 2:" << std::endl;
        for (auto x : *this)
            std::cout << x << "  ";
        b_vector::reserve(GetFileSize(filename) / sizeof(T));
    }
};

int main(int argc, char* argv[]) {
    std::chrono::system_clock::time_point begin_time =
        std::chrono::system_clock::now();

    mm_vector<int> v("H:\\save.txt");
    std::cout << std::endl << std::endl << "OUTPUT 2:" << std::endl;
    for (auto x : v)
        std::cout << x << "  ";

    std::chrono::system_clock::time_point end_time =
        std::chrono::system_clock::now();
    long long elapsed_miliseconds =
        std::chrono::duration_cast<std::chrono::milliseconds>(
            end_time - begin_time).count();
    std::cout << "Duration (min:seg:mili): " << std::setw(2)
              << std::setfill('0') << (elapsed_miliseconds / 60000) << ":"
              << std::setw(2) << std::setfill('0')
              << ((elapsed_miliseconds / 1000) % 60) << ":" << std::setw(2)
              << std::setfill('0') << (elapsed_miliseconds % 1000) << std::endl;
    std::cout << "Total milliseconds: " << elapsed_miliseconds << std::endl;

    return 0;
}

这篇关于stl向量中的内存映射文件存储的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！