Ruby流tar/gz [英] Ruby streaming tar/gz

问题描述

基本上，我想将内存中的数据流化为tar/gz格式(可能是将多个文件导入tar，但它绝不应该触摸硬盘，只能进行流化处理！)，然后再将它们以其他方式进行流化处理(我的HTTP请求正文中)情况).

Basically I want to stream data from memory into a tar/gz format (possibly multiple files into the tar, but it should NEVER TOUCH THE HARDDRIVE, only streaming!), then stream them somewhere else (an HTTP request body in my case).

任何人都知道可以执行此操作的现有库吗? Rails中有东西吗?

Anyone know of an existing library that can do this? Is there something in Rails?

libarchive-ruby 只是一个C包装程序，似乎它将非常适用于平台-依赖(文档希望您将其作为安装步骤进行编译?！).

libarchive-ruby is only a C wrapper and seems like it would be very platform-dependent (the docs want you to compile as an installation step?!).

解决方案:

require 'zlib'
require 'rubygems/package'

tar = StringIO.new

Gem::Package::TarWriter.new(tar) { |writer|
  writer.add_file("a_file.txt", 0644) { |f| 
    (1..1000).each { |i| 
      f.write("some text\n")
    }
  }
  writer.add_file("another_file.txt", 0644) { |f| 
    f.write("some more text\n")
  }
}
tar.seek(0)

gz = Zlib::GzipWriter.new(File.new('this_is_a_tar_gz.tar.gz', 'wb'))  # Make sure you use 'wb' for binary write!
gz.write(tar.read)
tar.close
gz.close

就是这样！您可以将GzipWriter中的文件与任何IO交换出去，以保持其流式传输. dw11wtq的饼干！

That's it! You can swap out the File in the GzipWriter with any IO to keep it streaming. Cookies for dw11wtq!

推荐答案

看看rubygems中的TarWriter类:

Take a look at the TarWriter class in rubygems: http://rubygems.rubyforge.org/rubygems-update/Gem/Package/TarWriter.html it just operates on an IO stream, which may be a StringIO.

tar = StringIO.new

Gem::Package::TarWriter.new(tar) do |writer|
  writer.add_file("hello_world.txt", 0644) { |f| f.write("Hello world!\n") }
end

tar.seek(0)

p tar.read #=> mostly padding, but a tar nonetheless

如果需要在tarball中进行目录布局，它还提供了添加目录的方法.

It also provides methods to add directories if you need a directory layout in the tarball.

作为参考，您可以使用IO.popen实现gzip压缩，只需将数据输入/输出到系统进程中即可.

For reference, you could achieve the gzipping with IO.popen, just piping the data in/out of the system process:

http://www.ruby- doc.org/core-1.9.2/IO.html#method-c-popen

gzip压缩本身看起来像这样:

The gzipping itself would look something like this:

gzippped_data = IO.popen("gzip", "w+") do |gzip|
  gzip.puts "Hello world!"
  gzip.close_write
  gzip.read
end
# => "\u001F\x8B\b\u0000\xFD\u001D\xA2N\u0000\u0003\xF3H\xCD\xC9\xC9W(\xCF/\xCAIQ\xE4\u0002\u0000A䩲\r\u0000\u0000\u0000"

这篇关于Ruby流tar/gz的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！