使用boto从S3逐行读取文件? [英] Read a file line by line from S3 using boto?

问题描述

我在S3中有一个csv文件，我正在尝试读取标题行以获取大小(这些文件是由我们的用户创建的，因此它们几乎可以是任何大小).有没有办法使用boto做到这一点?我以为也许可以给我们一个python BufferedReader，但是我不知道如何从S3键打开流.任何建议将是巨大的.谢谢！

I have a csv file in S3 and I'm trying to read the header line to get the size (these files are created by our users so they could be almost any size). Is there a way to do this using boto? I thought maybe I could us a python BufferedReader, but I can't figure out how to open a stream from an S3 key. Any suggestions would be great. Thanks!

推荐答案

看来，boto具有可以执行此操作的read()函数.这是一些对我有用的代码:

It appears that boto has a read() function that can do this. Here's some code that works for me:

>>> import boto
>>> from boto.s3.key import Key
>>> conn = boto.connect_s3('ap-southeast-2')
>>> bucket = conn.get_bucket('bucket-name')
>>> k = Key(bucket)
>>> k.key = 'filename.txt'
>>> k.open()
>>> k.read(10)
'This text '

对read(n)的调用将返回对象的下n个字节.

The call to read(n) returns the next n bytes from the object.

当然，这不会自动返回标题行"，但是您可以使用足够大的数字来调用它，以最少返回标题行.

Of course, this won't automatically return "the header line", but you could call it with a large enough number to return the header line at a minimum.

这篇关于使用boto从S3逐行读取文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！