如何按创建日期从 AWS S3 查询项目 [英] How to query items from AWS S3 by date created

问题描述

我想通过将项目添加到 S3 的日期/时间来查询存储桶中特定子目录中 S3 中的项目.我一直无法找到任何明确的文档，所以我想知道如何实现?

I want to query items from S3 within a specific subdirectory in a bucket by the date/time that they were added to S3. I haven't been able to find any explicit documentation around this, so I'm wondering how it can be accomplished?

我想要执行的查询类型如下...

The types of queries I want to perform look like this...

1. 返回images/user1/目录下S3存储桶images中最近创建的文件的URL
2. 返回在images/目录下的S3存储桶images中日期时间X和日期时间Y之间创建的所有项目的URL用户 1

1. Return the URL of the most recently created file in S3 bucket images under the directory images/user1/
2. Return the URLs of all items created between datetime X and datetime Y in the S3 bucket images under the directory images/user1

推荐答案

更新 3/19/2019

显然 s3api 可以让您轻松完成此操作

Apparently s3api allows you to do this quite easily

一种解决方案可能是使用 s3api.如果您的对象少于 1000 个，它很容易工作，否则您需要使用分页.

One solution would probably to use the s3api. It works easily if you have less than 1000 objects, otherwise you need to work with pagination.

s3api 可以列出所有对象，并且有一个属性用于 s3 中导入的键的 lastmodified 属性.然后可以对其进行排序，查找日期之后或之前的文件，匹配日期...

s3api can list all objects and has a property for the lastmodified attribute of keys imported in s3. It can then be sorted, find files after or before a date, matching a date ...

运行此类选项的示例

1. 给定日期的所有文件

1. all files for a given date

DATE=$(date +%Y-%m-%d)
aws s3api list-objects-v2 --bucket test-bucket-fh --query 'Contents[? 
contains(LastModified, `$DATE`)]'

某个日期之后的所有文件

all files after a certain date

export YESTERDAY=`date -v-1w +%F`
aws s3api list-objects-v2 --bucket test-bucket-fh --query 'Contents[? 
LastModified > `$YESTERDAY`)]'

s3api 将返回一些元数据，以便您可以过滤特定元素

s3api will return a few metadata so you can filter for specific elements

DATE=$(date +%Y-%m-%d)
aws s3api list-objects-v2 --bucket test-bucket-fh --query 'Contents[?contains(LastModified, `$DATE`)].Key'

<小时>

旧答案

AWS-SDK/CLI 真的应该实现某种按日期检索标志，它会让生活更轻松和>更便宜.

AWS-SDK/CLI really should implement some sort of retrieve-by-date flag, it would make life easier and cheaper.

如果您没有使用日期为文件添加前缀/标签，您可能还想尝试使用标志

If you have not prefixed/labelled your files with the dates, you may also want to try using the flag

--start-after (string)

如果您知道要开始列出的最新文件，您可以使用带有 --start-after 标志的 list-objects-v2 命令.

If you know the latest file you want to start listing from, you can use the list-objects-v2 command with the --start-after flag.

StartAfter 是您希望 Amazon S3 从哪里开始列出.Amazon S3 在此指定键之后开始列出.StartAfter 可以是存储桶中的任何键"

所以 --start-after 会不断地获取你的对象，所以如果你想限制项目的数量，试着指定一个 --max-items 标志.

So --start-after will continually get your objects, so if you would like to limit the number of items try specifying a --max-items flag.

https://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects-v2.html

这篇关于如何按创建日期从 AWS S3 查询项目的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！