从S3使用博托解压my_file.zip拉 [英] Unzip my_file.zip pulled from s3 using boto
问题描述
我想使用博托
打开的.zip
文件我在 S3
。我想直接用数据进行工作,我想避免创建临时文件。
I am trying to use boto
to open a .zip
file I have in s3
. I am trying to work with the data directly, I want to avoid creating temporary files.
In [201]: import StringIO
In [202]: import boto
In [203]: conn = boto.connect_s3()
In [204]: my_bucket = conn.get_bucket('my_bucket')
In [205]: my_list = [ele for ele in my_bucket.list('my_file.zip')]
In [206]: f = StringIO.StringIO()
In [207]: my_list[0].get_file(f)
In [208]: f.seek(0)
如果该文件没有被压缩,我只想用:
If the file was not zipped I would just use:
my_content = my_list[0].get_contents_as_string()
但因为它是压缩,我收到的垃圾。
but since it is zipped, I am getting garbage.
这个问题的回答做什么,我想(我借了一点使用了拉链
我试图从它),使用 GZIP
,但我无法找到任何东西。我试图用 zipfileZipFile
,而读
,提取物
和 extractall
方法似乎没有做我想做的。
An answer to this question does what I want (I borrowed a bit of my attempt from it) using gzip
, but I can't find anything using for zip
. I tried using zipfileZipFile
, but read
, extract
and extractall
methods don't seem to do what I want.
推荐答案
您应该看看Python模块GZIP:
You should look into the python module gzip :
https://docs.python.org/2/library/gzip.html
您应该能够StringIO的用gzip。
you should be able to stringIO with gzip. .
from boto.s3.connection import S3Connection
import gzip
from StringIO import StringIO
S3Conn = S3Connection() # assuming your .boto has been setup
Bucket = S3Conn.get_bucket('my_bucket')
my_list = [gzip.GzipFile(fileobj=(StringIO.StringIO(ele.get_contents_as_string()))) for ele in Bucket.list()]
#for readability I pulled this out
for item in my_list:
item.read()
有关可读性名单COM prehension或许应该被打破了 - 但我跟你原来的帖子比较。
for readability the list comprehension should probably be broken up - but I followed your original posting to compare.
祝你好运!
这篇关于从S3使用博托解压my_file.zip拉的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!