如何处理C ++与规模10亿的数组？ [英] How to handle an array with size 1,000,000,000 in C++?

问题描述

我需要处理3D多维数据集数据。其元件的数目可以是几十亿。我明白我不能分配在Windows上的内存。所以，我想基于磁盘的操作，在进程内数据库。有没有更好的方式来做到这一点？也许东西在推动？

I need to handle 3D cube data. Its number of elements can be several billions. I understand I can't allocate that much memory on Windows. So I am thinking disk-based operations with in-process database. Is there any better way to do this? Maybe something in boost?

更新：我最终要提供地块浏览功能

Update: I will eventually have to provide browsing functionality with plots.

UPDATE2：下面的文章似乎是使用内存映射文件一个很好的解决方案。我会尝试并再次更新。的http://www.$c$cproject.com/Articles/26275/Using-memory-mapped-files-to-conserve-physical-mem

Update2: The following article seemed to be a good solution using memory mapped file. I will try it and update again. http://www.codeproject.com/Articles/26275/Using-memory-mapped-files-to-conserve-physical-mem

推荐答案

你能也许更有效地存储数据（读宾利编程珠玑），它是稀疏数据？！

Can you perhaps store the data more efficiently (read "Programming Pearls" by Bentley), is it sparse data?!

如果不是，内存映射文件（MMF）是你的朋友，让你映射的MMF块到内存中，您可以访问像任何其他内存。

If not, memory mapped files (MMF) are your friend and allow you to map chunks of MMF into memory that you can access like any other memory.

使用 的CreateFileMapping 和的 MapViewOfFile 来一大块映射到你的进程。

Use CreateFileMapping and MapViewOfFile to map a chunk into your process.

这篇关于如何处理C ++与规模10亿的数组？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！