S3列表文件比 [英] S3 list files newer than

问题描述

我们有一个S3存储桶，以及一个包含所有访问日志的目录.我想列出比特定时间/日期新的文件.有什么办法可以实现?

We have an S3 bucket, and a directory with all our access logs. I would like to list files newer than a specific time/date. Any way this can be achieved?

该目录有超过100,000个文件，获取整个列表并遍历所有文件似乎效率不高.

The directory has over 100,000 files and it seems inefficient to fetch the entire list and iterate over them all.

提前谢谢！

推荐答案

好的.将对您到目前为止分享的内容做出一些假设.
如果这些是ELB生成的访问文件，则名称格式具有结构，如此处

Okay. Going to make a couple of assumption made on what you've shared so far.
If these are the access files generated by ELB, there is a structure to the format of the name, as documented here http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/access-log-collection.html

如果这些不是ELB日志，那么我将通过S3生命周期策略来解决此问题，如下所述:

If these are not the ELB logs, the way I would solve this is with an S3 Lifecycle Policy, as described here: http://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html
In this case objects older than the duration you set could be expired (i.e. deleted or moved to glacier.

这篇关于S3列表文件比的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！