有人可以提供使用boost iostreams搜索，读取和写入一个> 4GB文件的示例 [英] Can someone provide an example of seeking, reading, and writing a >4GB file using boost iostreams

问题描述

我读过，boost iostreams据说支持64位访问大文件半便携式的方式。他们的常见问题提供 64位偏移函数，但没有关于如何使用它们的示例。有没有人使用这个库来处理大文件？

谢谢。

div class =h2_lin>解决方案

简短答案

只需添加

  #include< boost / iostreams / seek.hpp> 
  

并使用 seek 函数

  boost :: iostreams :: seek（device，offset，whence）; 
  

其中

• 设备是一个文件，流，streambuf或任何可转换为 seekable 的对象;


• offset 是 stream_offset ;


• whence 是 BOOST_IOS :: beg ， BOOST_IOS :: cur 或 BOOST_IOS :: end 。

seek的返回值是 std :: streampos 类型，并且可以使用 c转换为 stream_offset http://www.boost.org/doc/libs/1_42_0/libs/iostreams/doc/functions/positioning.html#position_to_offset> position_to_offset 函数。

示例

这是一个冗长乏味的重复示例，显示如何打开两个文件，寻找离开大小> 4GB，并在它们之间复制数据。

警告：此代码将创建非常大的文件（几GB）。在支持稀疏文件的OS /文件系统上尝试此示例。 Linux是确定;

  / * 
 *警告：这会创建非常大的文件（几GB）
 *，除非您的操作系统/文件系统支持稀疏文件。 
 * / 
 #include< boost / iostreams / device / file.hpp> 
 #include< boost / iostreams / positioning.hpp> 
 #include< cstring> 
 #include< iostream> 
 
使用boost :: iostreams :: file_sink; 
 using boost :: iostreams :: file_source; 
 using boost :: iostreams :: position_to_offset; 
 using boost :: iostreams :: seek; 
 using boost :: iostreams :: stream_offset; 
 
 static const stream_offset GB = 1000 * 1000 * 1000; 
 
 void setup（）
 {
 file_sink out（file1，BOOST_IOS :: binary）; 
 const char * greetings [] = {Hello，Boost，World}; 
 for（int i = 0; i <3; i ++）{
 out.write（greetings [i]，5）; 
 seek（out，7 * GB，BOOST_IOS :: cur）; 
} 
} 
 
 void copy_file1_to_file2（）
 {
 file_source in（file1，BOOST_IOS :: binary）; 
 file_sink out（file2，BOOST_IOS :: binary）; 
 stream_offset off; 
 
 off = position_to_offset（seek（in，-5，BOOST_IOS :: end））; 
 std :: cout<< in：seek< off<< std :: endl; 
 
 for（int i = 0; i <3; i ++）{
 char buf [6]; 
 std :: memset（buf，'\0'，sizeof buf）; 
 
 std :: streamsize nr = in.read（buf，5）; 
 std :: streamsise nw = out.write（buf，5）; 
 std :: cout<< read：\<< buf<<\（<< nr<<），
< written：（<< nw<<）< std :: endl; 
 
 off = position_to_offset（seek（in， - （7 * GB + 10），BOOST_IOS :: cur））; 
 std :: cout<< in：seek< off<< std :: endl; 
 off = position_to_offset（seek（out，7 * GB，BOOST_IOS :: cur））; 
 std :: cout<< out：seek<< off<< std :: endl; 
} 
} 
 
 int main（）
 {
 setup（）; 
 copy_file1_to_file2（）; 
} 
  

I have read that boost iostreams supposedly supports 64 bit access to large files semi-portable way. Their FAQ mentions 64 bit offset functions, but there is no examples on how to use them. Has anyone used this library for handling large files? A simple example of opening two files, seeking to their middles, and copying one to the other would be very helpful.

Thanks.

解决方案

Short answer

Just include

#include <boost/iostreams/seek.hpp>

and use the seek function as in

boost::iostreams::seek(device, offset, whence);

where

• device is a file, stream, streambuf or any object convertible to seekable;
• offset is a 64-bit offset of type stream_offset;
• whence is BOOST_IOS::beg, BOOST_IOS::cur or BOOST_IOS::end.

The return value of seek is of type std::streampos, and it can be converted to a stream_offset using the position_to_offset function.

Example

Here is an long, tedious and repetitive example, which shows how to open two files, seek to offstets >4GB, and copying data between them.

WARNING: This code will create very large files (several GB). Try this example on an OS/file system which supports sparse files. Linux is ok; I did not test it on other systems, such as Windows.

/*
 * WARNING: This creates very large files (several GB)
 * unless your OS/file system supports sparse files.
 */
#include <boost/iostreams/device/file.hpp>
#include <boost/iostreams/positioning.hpp>
#include <cstring>
#include <iostream>

using boost::iostreams::file_sink;
using boost::iostreams::file_source;
using boost::iostreams::position_to_offset;
using boost::iostreams::seek;
using boost::iostreams::stream_offset;

static const stream_offset GB = 1000*1000*1000;

void setup()
{
    file_sink out("file1", BOOST_IOS::binary);
    const char *greetings[] = {"Hello", "Boost", "World"};
    for (int i = 0; i < 3; i++) {
        out.write(greetings[i], 5);
        seek(out, 7*GB, BOOST_IOS::cur);
    }
}

void copy_file1_to_file2()
{
    file_source in("file1", BOOST_IOS::binary);
    file_sink out("file2", BOOST_IOS::binary);
    stream_offset off;

    off = position_to_offset(seek(in, -5, BOOST_IOS::end));
    std::cout << "in: seek " << off << std::endl;

    for (int i = 0; i < 3; i++) {
        char buf[6];
        std::memset(buf, '\0', sizeof buf);

        std::streamsize nr = in.read(buf, 5);
        std::streamsize nw = out.write(buf, 5);
        std::cout << "read: \"" << buf << "\"(" << nr << "), "
                  << "written: (" << nw << ")" << std::endl;

        off = position_to_offset(seek(in, -(7*GB + 10), BOOST_IOS::cur));
        std::cout << "in: seek " << off << std::endl;
        off = position_to_offset(seek(out, 7*GB, BOOST_IOS::cur));
        std::cout << "out: seek " << off << std::endl;
    }
}

int main()
{
    setup();
    copy_file1_to_file2();
}

这篇关于有人可以提供使用boost iostreams搜索，读取和写入一个> 4GB文件的示例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！