如何有效地将数据插入大文件 [英] How to Insert Data into Large Files Efficiently

问题描述

亲爱的所有人，
目前，我实现了一个用于从Internet下载文件的下载器.并且文件已分为几个块，每个块包括数据，数据长度和要写入文件的起始地址.但是，正如您所知，我们得到的块不是顺序的，因此它不能顺序地写入文件.对于这种情况，如果文件很大(例如超过2G)，并且如果我是第一次将块放在2G位置，并且如果直接将其写入文件，则将花费大量时间.
如何有效地写入此类数据?
最好的问候，
Mike

Dear all,
Currently I implement a downloader which uses for download file from internet. And the file has been divided into several blocks, every block includes data, the length of data and the start address to be written into file. But as you know, the block we gets it isn''t sequence, so it can''t write into file sequencely. For the situation, if the file is very large, like more than 2G, and if the first time I get the block at 2G position, and if it write into file directly, it costs a lot time.
How can it writes such data efficiently?
Best regards,
Mike

推荐答案

如果您在Windows上进行编程，则可能会找到

If you''re programming on windows you might find sparse files[^] do what you want.

For most purposes the file looks like it''s full of zeroes but the unoccupied bits don''t take up any diskspace.

Cheers,

Ash

这个天真的代码(警告:错误处理非常差)

This naive code (warning: very bad error-handling)

#include <stdio.h>
int main()
{
  unsigned char a[1024];
  FILE * fp = fopen(filepath, "wb");
  if ( !fp ) return -1;
  if (fseek(fp, 1000000000L,SEEK_SET) ) return -2;
  if (fwrite(a, 1, sizeof(a), fp) != sizeof(a) ) return -3;
  fclose(fp);
}

在我的系统上大约需要25英寸.如果这么长的时间是不可接受的，则将接收到的块保存在临时文件中，最终在后台线程中重新构成整个大文件.
:)

takes circa 25" on my system. If such a long time is not acceptable then save the received blocks on temporary files, eventually recomposing the complete big file in a background thread.
:)

这篇关于如何有效地将数据插入大文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！