将 R 数据作为 csv 直接写入 s3 [英] Write R data as csv directly to s3

问题描述

我希望能够将数据从 data.frame data.table 对象作为 csv 文件直接写入 AWS s3 中的存储桶，而无需写入首先使用 AWS CLI 进行磁盘处理.

I would like to be able to write data directly to a bucket in AWS s3 from a data.frame data.table object as a csv file without writing it to disk first using the AWS CLI.

obj.to.write.s3 <- data.frame(cbind(x1=rnorm(1e6),x2=rnorm(1e6,5,10),x3=rnorm(1e6,20,1)))

目前我先写入 csv，然后上传到现有存储桶，然后使用以下方法删除文件:

at the moment I write to csv first then upload to an existing bucket then remove the file using:

fn <- 'new-file-name.csv'
write.csv(obj.to.write.s3,file=fn)
system(paste0('aws s3 ',fn,' s3://my-bucket-name/',fn))
system(paste0('rm ',fn))

我想要一个直接写入 s3 的函数?这可能吗?

I would like a function that writes directly to s3? is that possible?

推荐答案

在 aws.s30.2.2 添加了 s3write_using()(和 s3read_using())函数.

In aws.s3 0.2.2 the s3write_using() (and s3read_using()) functions were added.

它们使事情变得更简单:

They make things much simpler:

s3write_using(iris, FUN = write.csv,
                    bucket = "bucketname",
                    object = "objectname")

这篇关于将 R 数据作为 csv 直接写入 s3的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！