如何使用Ruby将文件内容从S3存储桶下载到内存中 [英] How to download the content of a file from S3 bucket into memory with Ruby

问题描述

我在Amazon AWS S3中有一个存储桶，其中有一个名为users.csv的文件.

I have a bucket in Amazon AWS S3 which has a file in it called users.csv.

如何使用Ruby将S3存储桶中的该文件的内容加载到内存中，以便我可以对其进行解析?

How can I load the content of this file from the S3 bucket into memory with Ruby so I can parse it?

这是我的代码:

require 'aws-sdk'
s3 = Aws::S3::Resource.new(region: 'us-west-1')
resp = s3.list_objects(bucket: 'bucket-name', max_keys: 1)
resp.contents.each do |object|
    puts #{object.value}
end

当我在IRB中尝试此操作时，我得到了:

When I tried this in IRB I got:

struct Aws::S3::Types::Object key="users.csv", last_modified=2017-11-15 19:10:28 UTC, etag="\"9a3d50c07aa4aa6976037ce774294a26\"", size=101, storage_class="STANDARD", owner=struct Aws::S3::Types::Owner display_name="owner-name", id="42093cfa4ccb23a8156cdab8500a41a10bdbf90deebb0ee8a3b340dd1e0c3622"

如何解析users.csv的内容?

推荐答案

来自AWS文档:

将对象下载到内存中

Downloading Objects into Memory

对于小对象，获取对象并将其在Ruby进程中可用可能很有用.如果你 不为下载指定:target，则整个对象已加载 进入内存并放入StringIO对象.

For small objects, it can be useful to get an object and have it available in your Ruby processes. If you do not specify a :target for the download, the entire object is loaded into memory into a StringIO object.

def import_from_s3 

  #initiate the client
  s3 = Aws::S3::Client.new({
      region:            region,
      access_key_id:     key_id,
      secret_access_key: secret
  })

  #Get the object
  resp = s3.get_object(bucket: bucket, key: key)

 resp.body
 #=> #<StringIO ...> 

 resp.body.read
 #=> '...'

在StringIO上调用#read或#string以使主体成为String对象.

Call #read or #string on the StringIO to get the body as a String object.

有关更多信息，请参见"使用适用于Ruby的AWS开发工具包从Amazon S3下载对象.

For more information see "Downloading Objects from Amazon S3 using the AWS SDK for Ruby".

这篇关于如何使用Ruby将文件内容从S3存储桶下载到内存中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！