使用上次修改日期条件删除多个s3存储桶文件 [英] Delete multiple s3 bucket files with Last Modified date condition

问题描述

如何删除具有最后修改日期日期条件的多个S3文件?

我在s3上具有此文件夹结构.

• dentca-lab-dev-sample
  • 2019-03-13
    • file1上次修改时间:2019年3月13日下午2:34:06 GMT-0700
    • file2上次修改时间:2019年3月13日下午3:18:01 GMT-0700
    • file3上次修改时间:2019年3月13日下午2:34:30 GMT-0700
    • file4上次修改时间:2019年3月13日下午2:32:40 GMT-0700

，并希望删除小于Mar 13, 2019 2:34:30 PM

的文件(这只是一个示例)

所以我制作了这个bash脚本，但是它不起作用.

aws s3 ls --recursive s3://dentca-lab-dev-sample/2019-03-13/ | awk '$1 <= "2019-03-13 14:34:30" {print $4}'

** ls仅用于测试.将其更改为rm

我也有用于测试的脚本

aws s3 ls --recursive s3://dentca-lab-dev-sample/2019-03-13/

输出:

2019-03-13 14:34:06   11656584 2019-03-13/mandibular.stl
2019-03-13 15:18:01   11969184 2019-03-13/maxillary.stl
2019-03-13 14:34:30    9169657 2019-03-13/obj.obj
2019-03-13 14:32:40   15690284 2019-03-13/upperAIO_50005.stl

，但是当我执行awk使条件不起作用时.可能是因为$1仅捕获了这种折衷2019-03-13并将其即时编译为2019-03-13 14:34:30

也尝试这样做. awk '$1 $2 <= "2019-03-13 14:34:30" {print $4}'抓住第二个论点，但仍然一无所获.这是我的第一个bash b .

谢谢！我将其作为参考. aws cli s3存储桶删除具有日期条件的对象

解决方案

您可以使用它来获取给定日期前带有LastModified的对象的列表:

aws s3api list-objects --bucket my-bucket --query "Contents[?LastModified<='2019-03-13'].[Key]" --output text

请注意，它使用的是s3api而不是s3，它可以访问更多信息.

然后您可以获取结果并将其泵入aws s3 rm以删除对象.

坦率地说，如果您希望获得这样的细粒度，我建议您使用Python而不是bash.就像这样:

import boto3

s3 = boto3.client('s3', region_name='ap-southeast-2')
response = s3.list_objects_v2(Bucket='my-bucket')

keys_to_delete = [{'Key': object['Key']} for object in response['Contents'] if object['LastModified'] < datetime(2019, 3, 13)]
s3.delete_objects(Bucket='my-bucket', Delete={'Objects': keys_to_delete})

How do you delete multiple S3 files with Last Modified date condition?

I have this folder structure on s3.

• dentca-lab-dev-sample
  • 2019-03-13
    • file1 Last modified: Mar 13, 2019 2:34:06 PM GMT-0700
    • file2 Last modified: Mar 13, 2019 3:18:01 PM GMT-0700
    • file3 Last modified: Mar 13, 2019 2:34:30 PM GMT-0700
    • file4 Last modified: Mar 13, 2019 2:32:40 PM GMT-0700

and wanted to delete a file (this is just a sample) less than Mar 13, 2019 2:34:30 PM

and so I made this bash script but its not working.

aws s3 ls --recursive s3://dentca-lab-dev-sample/2019-03-13/ | awk '$1 <= "2019-03-13 14:34:30" {print $4}'

** ls is just for testing. will change it to rm

I also have this script for testing

aws s3 ls --recursive s3://dentca-lab-dev-sample/2019-03-13/

output:

2019-03-13 14:34:06   11656584 2019-03-13/mandibular.stl
2019-03-13 15:18:01   11969184 2019-03-13/maxillary.stl
2019-03-13 14:34:30    9169657 2019-03-13/obj.obj
2019-03-13 14:32:40   15690284 2019-03-13/upperAIO_50005.stl

but when I do the awk to make condition doesn't work. Maybe because $1 only catches this arugment 2019-03-13 and im compering it to 2019-03-13 14:34:30

also tried doing this. awk '$1 $2 <= "2019-03-13 14:34:30" {print $4}' to catch the second argument but still got nothing. Its my first to make a bash btw.

Thank you! I have this as reference btw. aws cli s3 bucket remove object with date condition

解决方案

You can use this to obtain a list of objects with a LastModified before a given date:

aws s3api list-objects --bucket my-bucket --query "Contents[?LastModified<='2019-03-13'].[Key]" --output text

Note that it uses s3api rather than s3, which has access to more information.

You could then take the results and pump them into aws s3 rm to delete the objects.

Frankly, if you wish to get fine-grained like this, I would recommend using Python instead of bash. It would be something like:

import boto3

s3 = boto3.client('s3', region_name='ap-southeast-2')
response = s3.list_objects_v2(Bucket='my-bucket')

keys_to_delete = [{'Key': object['Key']} for object in response['Contents'] if object['LastModified'] < datetime(2019, 3, 13)]
s3.delete_objects(Bucket='my-bucket', Delete={'Objects': keys_to_delete})

这篇关于使用上次修改日期条件删除多个s3存储桶文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！