在OpenCV中有效地将大Mat装载到内存中 [英] Efficiently load a large Mat into memory in OpenCV

问题描述

有什么更有效的方法来加载一个大的Mat对象到内存比OpenCV中的FileStorage方法？

Is there a more efficient way to load a large Mat object into memory than the FileStorage method in OpenCV?

我有一个大的Mat 192列和1百万行我想在本地存储在一个文件，并加载到内存中，然后我的应用程序启动。使用FileStorage没有问题，但我想知道是否存在一个更有效的方法来做到这一点。此时，使用Visual Studio中的调试模式和发布模式下大约3分钟将Mat加载到内存中需要大约5分钟，数据文件的大小约为1.2GB。

I have a large Mat with 192 columns and 1 million rows I want to store locally in a file and load into memory then my application starts. There is no problem using the FileStorage, but I was wondering if there exists a more efficient method to do this. At the moment it takes about 5 minutes to load the Mat into memory using the Debug mode in Visual Studio and around 3 minutes in the Release mode and the size of the data file is around 1.2GB.

FileStorage方法是唯一可用于执行此任务的方法吗？

Is the FileStorage method the only method available to do this task?

推荐答案

100x 加速？

二进制格式。您可以在下面的代码中使用 matwrite 和 matread 函数来实现。

You should save and load your images in binary format. You can do that with the matwrite and matread function in the code below.

我测试了从 FileStorage 和二进制文件加载，对于250K行，192列的较小映像，键入 CV_8UC1 我得到这些结果（时间以毫秒为单位）：

I tested both loading from a FileStorage and the binary file, and for a smaller image with 250K rows, 192 columns, type CV_8UC1 I got these results (time in ms):

// Mat: 250K rows, 192 cols, type CV_8UC1
Using FileStorage: 5523.45
Using Raw:         50.0879    

使用我获得的二进制模式（以ms为单位）的1M行和192列图像：

On a image with 1M rows and 192 cols using the binary mode I got (time in ms):

// Mat: 1M rows, 192 cols, type CV_8UC1
Using FileStorage: (can't load, out of memory)
Using Raw:         197.381

注

1. em>


2. 3分钟加载矩阵似乎太多了，即使对于 FileStorage 。

1. Never measure performance in debug.
2. 3 minutes to load a matrix seems way too much, even for FileStorages. However, you'll gain a lot switching to binary mode.

这里的函数 matwrite 和 matread 和测试：

#include <opencv2\opencv.hpp>
#include <iostream>
#include <fstream>

using namespace std;
using namespace cv;


void matwrite(const string& filename, const Mat& mat)
{
    ofstream fs(filename, fstream::binary);

    // Header
    int type = mat.type();
    int channels = mat.channels();
    fs.write((char*)&mat.rows, sizeof(int));    // rows
    fs.write((char*)&mat.cols, sizeof(int));    // cols
    fs.write((char*)&type, sizeof(int));        // type
    fs.write((char*)&channels, sizeof(int));    // channels

    // Data
    if (mat.isContinuous())
    {
        fs.write(mat.ptr<char>(0), (mat.dataend - mat.datastart));
    }
    else
    {
        int rowsz = CV_ELEM_SIZE(type) * mat.cols;
        for (int r = 0; r < mat.rows; ++r)
        {
            fs.write(mat.ptr<char>(r), rowsz);
        }
    }
}

Mat matread(const string& filename)
{
    ifstream fs(filename, fstream::binary);

    // Header
    int rows, cols, type, channels;
    fs.read((char*)&rows, sizeof(int));         // rows
    fs.read((char*)&cols, sizeof(int));         // cols
    fs.read((char*)&type, sizeof(int));         // type
    fs.read((char*)&channels, sizeof(int));     // channels

    // Data
    Mat mat(rows, cols, type);
    fs.read((char*)mat.data, CV_ELEM_SIZE(type) * rows * cols);

    return mat;
}

int main()
{
    // Save the random generated data
    {
        Mat m(1024*256, 192, CV_8UC1);
        randu(m, 0, 1000);

        FileStorage fs("fs.yml", FileStorage::WRITE);
        fs << "m" << m;

        matwrite("raw.bin", m);
    }

    // Load the saved matrix

    {
        // Method 1: using FileStorage
        double tic = double(getTickCount());

        FileStorage fs("fs.yml", FileStorage::READ);
        Mat m1;
        fs["m"] >> m1;

        double toc = (double(getTickCount()) - tic) * 1000. / getTickFrequency();
        cout << "Using FileStorage: " << toc << endl; 
    }

    {
        // Method 2: usign raw binary data
        double tic = double(getTickCount());

        Mat m2 = matread("raw.bin");

        double toc = (double(getTickCount()) - tic) * 1000. / getTickFrequency();
        cout << "Using Raw: " << toc << endl;
    }

    int dummy;
    cin >> dummy;

    return 0;
}

这篇关于在OpenCV中有效地将大Mat装载到内存中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！