从AWS S3存储桶读取的R reactFileFileReader [英] R reactiveFileReader reading from aws s3 bucket

问题描述

我可以使用以下代码从S3存储桶中读取csv

I can read a csv from my S3 bucket using the below code

aws.s3::s3read_using(read.csv,
                     stringsAsFactors=FALSE,
                     check.names=FALSE,
                     object=paste0(Sys.getenv("BUCKET_PREFIX"), "/a.csv"),
                     bucket = Sys.getenv("AWS_BUCKET_NAME"),
                     opts=bucket_opts
    )

我想将其更改为使用函数reactFileReader.我尝试了以下操作，但都没有成功，知道我在做什么错吗?

I want to change this to using the function reactiveFileReader. I tried the below with no success, any idea what I am doing wrong?

reactiveFileReader(
      intervalMillis = 10000,
      session= session,
      filePath = paste0(Sys.getenv("BUCKET_PREFIX"), "/a.csv"),
      readFunc = aws.s3::s3read_using,
      FUN = read.csv,
      stringsAsFactors=FALSE,
      check.names=FALSE,
      object=paste0(Sys.getenv("BUCKET_PREFIX"), "/a.csv"),
      bucket = Sys.getenv("AWS_BUCKET_NAME"),
      opts=bucket_opts
      )

推荐答案

我遇到了同样的问题，并通过一些解决方法解决了该问题:

I had the same problem and solved it with a bit of a workaround:

在global.R中，您会在闪亮的应用程序启动时读取数据.

in global.R you read your data at start of shiny app.

data_obj <- aws.s3::s3read_using(read.csv,
                     stringsAsFactors=FALSE,
                     check.names=FALSE,
                     object=paste0(Sys.getenv("BUCKET_PREFIX"), "/a.csv"),
                     bucket = Sys.getenv("AWS_BUCKET_NAME"),
                     opts=bucket_opts
    )

#bucket information
content_bucket <- aws.s3::get_bucket(bucket = Sys.getenv("AWS_BUCKET_NAME"))

在server.R中，您设置了一个计时器来检查S3上的数据是否已更改.

in server.R you set a timer to check if data on S3 changed.

  shiny::observe({

shiny::invalidateLater(3600e3) #timer for every hour to check the following

#temporary bucket information
bucket_content_check <- aws.s3::get_bucket(bucket = Sys.getenv("AWS_BUCKET_NAME"))

#reload data if timestamp differs
    if (bucket_content$Contents$LastModified != bucket_content_check$Contents$LastModified) {
      
data_obj <<- aws.s3::s3read_using(read.csv,
                     stringsAsFactors=FALSE,
                     check.names=FALSE,
                     object=paste0(Sys.getenv("BUCKET_PREFIX"), "/a.csv"),
                     bucket = Sys.getenv("AWS_BUCKET_NAME"),
                     opts=bucket_opts
    )

#overwrite global bucket_content for next check
content_bucket <<- aws.s3::get_bucket(bucket = Sys.getenv("AWS_BUCKET_NAME"))

message('S3 data reloaded')
      }

    message('S3 bucket checked')
})

这篇关于从AWS S3存储桶读取的R reactFileFileReader的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！