重命名具有随机目录结构的传入S3文件 [英] Rename an incoming S3 file with a random directory structure

问题描述

我有这个应用程序，它将文件发送到s3存储桶.不幸的是，我无法更改它在s3中发送到的路径，因此我必须找出一种获取此文件的方法.

I have this app that will send a file to a s3 bucket. unfortunately I cannot change the path it sends it to in s3 so I have to figure out a way to get this file.

mys3bucket:/apps/region/020-07-14T22:24:34Z/details.csv

mys3bucket: /apps/region/020-07-14T22:24:34Z/details.csv

您可以看到日期，该应用会将日期放入路径.我试图不对项目进行硬编码以使其更加灵活.

As you can see the date, the app places the date into the path. I am trying to not hard code items to make it more flexible.

我要做的是重命名details.csv文件并将其移动到同一s3存储桶中的另一个位置.基本上是它的永久位置.

what I want to do is get that details.csv file rename and move it to another location within the same s3 bucket. basically its permanent location.

我正在尝试的是这样的东西，但是显然它不适用于随机路径.我唯一可以为其创建变量的部分是:

what I was trying was something like this but it clearly will not work with the random path. the only piece that I can make a variable for is:

path =/apps/region/下一级是随机的，但报告名称始终相同.

path = /apps/region/ the next level is random, but the report name is always the same.

很明显我没有尝试正确的方法，但是到目前为止我还不确定.

clearly im not trying this the correct way but as of now I am not sure.

s3.Object( 'mys3bucket' ,'account3_details.csv').copy_from(CopySource='mys3bucket/apps/region/2020-07-14T22:24:34Z/details.csv')
s3.Object( 'mys3bucket','/apps/region/2020-07-14T22:24:34Z/details.csv').delete()

推荐答案

以下是我先前编写的函数中的一些代码.当由Amazon S3事件触发时，它将对象移动到新位置:

Here is some code from a function I previously wrote. When triggered by an Amazon S3 Event, it moves the object to a new location:

import boto3
import urllib

TARGET_BUCKET = 'my-bucket'
TARGET_PATH = 'foo/'

def lambda_handler(event, context):
    
    # Get incoming bucket and key
    source_bucket = event['Records'][0]['s3']['bucket']['name']
    source_key = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'])

    # Extract filename without path
    filename = ('/' + source_key).rsplit('/', 1)[1]

    # Copy object to different bucket
    s3_resource = boto3.resource('s3')
    copy_source = {
        'Bucket': source_bucket,
        'Key': source_key
    }
    s3_resource.Bucket(TARGET_BUCKET).Object(TARGET_PATH + filename).copy(copy_source)
    s3_resource.Bucket(source_bucket).Object(source_key).delete()

您需要修改确定复制对象的目标文件名(键)的逻辑.

You would need to modify the logic that determines the destination filename (Key) of the copied object.

这篇关于重命名具有随机目录结构的传入S3文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！