S3:如何在不下载完整文件的情况下进行部分读取/查找? [英] S3: How to do a partial read / seek without downloading the complete file?

问题描述

虽然它们类似于文件，但 Amazon S3 中的对象并不是真正的文件"，就像 S3 存储桶不是真正的目录一样.在 Unix 系统上，我可以使用 head 预览文件的前几行，无论它有多大，但我不能在 S3 上这样做.那么如何在 S3 上进行部分读取呢?

Although they resemble files, objects in Amazon S3 aren't really "files", just like S3 buckets aren't really directories. On a Unix system I can use head to preview the first few lines of a file, no matter how large it is, but I can't do this on a S3. So how do I do a partial read on S3?

推荐答案

S3 文件可能很大，但您不必为了读取前几个字节而获取整个文件.S3 API 支持 HTTP Range: 标头 (参见 RFC 2616)，它采用字节范围参数.

S3 files can be huge, but you don't have to fetch the entire thing just to read the first few bytes. The S3 APIs support the HTTP Range: header (see RFC 2616), which take a byte range argument.

只需向您的 S3 请求添加一个 Range: bytes=0-NN 标头，其中 NN 是请求读取的字节数，您将仅获取这些字节而不是读取整个字节文件.现在，您可以预览您留在 S3 存储桶中的 900 GB CSV 文件，而无需等待整个文件下载完毕.阅读 完整的 GET Object 文档 在亚马逊的开发者文档中.

Just add a Range: bytes=0-NN header to your S3 request, where NN is the requested number of bytes to read, and you'll fetch only those bytes rather than read the whole file. Now you can preview that 900 GB CSV file you left in an S3 bucket without waiting for the entire thing to download. Read the full GET Object docs on Amazon's developer docs.

这篇关于S3:如何在不下载完整文件的情况下进行部分读取/查找?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！