为什么我的程序无法将大量(> 2GB)保存到文件中? [英] Why can't my program save a large amount (>2GB) to a file?

问题描述

我很难弄清楚为什么我的程序不能将超过2GB的数据保存到文件中.我无法确定这是编程还是环境(OS)问题.这是我的源代码:

I am having trouble trying to figure out why my program cannot save more than 2GB of data to a file. I cannot tell if this is a programming or environment (OS) problem. Here is my source code:

#define _LARGEFILE_SOURCE
#define _LARGEFILE64_SOURCE
#define _FILE_OFFSET_BITS 64
#include <math.h>
#include <time.h>
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
/*-------------------------------------*/
//for file mapping in Linux
#include<fcntl.h>
#include<unistd.h>
#include<sys/stat.h>
#include<sys/time.h>
#include<sys/mman.h>
#include<sys/types.h>
/*-------------------------------------*/
#define PERMS 0600
#define NEW(type) (type *) malloc(sizeof(type))
#define FILE_MODE (S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH)

void write_result(char *filename, char *data, long long length){

int fd, fq;

  fd = open(filename, O_RDWR|O_CREAT|O_LARGEFILE, 0644);

  if (fd < 0) {
  perror(filename);
  return -1;
  }

  if (ftruncate(fd, length) < 0)
  {
  printf("[%d]-ftruncate64 error: %s/n", errno, strerror(errno));
  close(fd);
  return 0;
  }

  fq = write (fd, data,length);

  close(fd);

  return;

}

main()
{
long long offset = 3000000000; // 3GB
char * ttt;
ttt = (char *)malloc(sizeof(char) *offset);
printf("length->%lld\n",strlen(ttt));  // length=0
memset (ttt,1,offset);
printf("length->%lld\n",strlen(ttt));  // length=3GB
write_result("test.big",ttt,offset);
return 1;
}

根据我的测试，该程序可以生成一个大于2GB的文件，并且还可以分配这么大的内存. 当我尝试将数据写入文件时发生了奇怪的事情.我检查了文件，它是空的，应该用1填充.

According to my test, the program can generate a file large than 2GB and can allocate such large memory as well. The weird thing happened when I tried to write data into the file. I checked the file and it is empty, which is supposed to be filled with 1.

有人能帮助我吗?

推荐答案

我要在这里走出去，说你的问题可能出在memset()上.

I'm going to go out on a limb here and say that your problem may lay in memset().

我认为最好的做法是在memset()之后，

The best thing to do here is, I think, after memset() ing it,

for (unsigned long i = 0; i < 3000000000; i++) {
  if (ttt[i] != 1) { printf("error in data at location %d", i); break; }
}

一旦您确认要写入的数据正确无误，则应考虑编写一个较小的文件(例如1GB)，看看是否存在相同的问题.消除每个可能的变量，您将找到答案.

Once you've validated that the data you're trying to write is correct, then you should look into writing a smaller file such as 1GB and see if you have the same problems. Eliminate each and every possible variable and you will find the answer.

这篇关于为什么我的程序无法将大量(> 2GB)保存到文件中?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！