从 R 访问 S3 存储桶 [英] To access S3 bucket from R

问题描述

我在 AWS 上的 EC2 实例上设置了 R.我有几个 csv 文件上传到 S3 存储桶.我想知道是否有办法从 R 访问 S3 存储桶中的 csv 文件.

I have set-up R on an EC2 Instance on AWS. I have few csv files uploaded into a S3 bucket. I was wondering if there is a way to access the csv files in the S3 bucket from R.

任何帮助/指针将不胜感激.

Any help/pointers would be appreciated.

推荐答案

查看 cloudyr aws.s3 包 (https://github.com/cloudyr/aws.s3)，它可能会满足您的需求.不幸的是(在撰写本文时)，这个包还处于早期阶段&有点不稳定.

Have a look at the cloudyr aws.s3 package (https://github.com/cloudyr/aws.s3), it might do what you need. Unfortunately (at time of writing), this package is quite early stage & a little unstable.

我仅使用 R 的 system() 命令调用 AWS CLI 就取得了很好的成功.这相对容易上手，非常强大且支持良好.

I've had good success simply using R's system() command to make a call to the AWS CLI. This is relatively easy to get started on, very robust and very well supported.

1. 从这里开始:http://aws.amazon.com/cli/
2. 使用 S3 API 列出对象:http://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects.html
3. 使用 S3 API 获取对象:http://docs.aws.amazon.com/cli/latest/reference/s3api/get-object.html

因此，例如，在命令行上尝试以下操作:

So, for example, on command-line try following:

pip install awscli
aws configure
aws s3 help
aws s3api list-objects --bucket some-bucket --query 'Contents[].{Key: Key}'
aws s3api get-object --bucket some-bucket --key some_file.csv new_file_name.csv

在 R 中，可以这样做:

In R, can just do something like:

system("aws s3api list-objects --bucket some-bucket --query 'Contents[].{Key: Key}' > my_bucket.json")

这篇关于从 R 访问 S3 存储桶的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！