将S3文件检索为对象，而不是下载到绝对系统路径 [英] Retrieve S3 file as Object instead of downloading to absolute system path

问题描述

我刚刚开始学习和使用S3，请阅读文档.实际上，我没有找到任何东西可以将文件提取到对象中，而不是从S3下载?如果可能的话，或者我错过了什么?

I just started learning and using S3, read the docs. Actually I didn't find anything to fetch the file into an object instead of downloading it from S3? if this could be possible, or I am missing something?

实际上，我想在下载文件后避免额外的IO.

Actually I want to avoid additional IO after downloading the file.

推荐答案

您可以使用StringIO并使用get_contents_as_string从S3获取文件内容，如下所示:

You could use StringIO and get file content from S3 using get_contents_as_string, like this:

import pandas as pd
import StringIO
from boto.s3.connection import S3Connection

AWS_KEY = 'XXXXXXDDDDDD'
AWS_SECRET = 'pweqory83743rywiuedq'
aws_connection = S3Connection(AWS_KEY, AWS_SECRET)
bucket = aws_connection.get_bucket('YOUR_BUCKET')

fileName = "test.csv"

content = bucket.get_key(fileName).get_contents_as_string()
reader = pd.read_csv(StringIO.StringIO(content))

这篇关于将S3文件检索为对象，而不是下载到绝对系统路径的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！