How to list the contents of Amazon S3 by modified date

Question

Most of the time it so happens that we load so many files in a common S3 bucket due to which it becomes hard to figure out data in it.

Is there a way where i can view objects uploaded on a particular date?

Answer 1

One easy solution would be probably to use the s3api. It works easily if you have less than 1000 objects, otherwise you might have to use pagination.

s3api lists all objects and has a property for the lastmodified attribute of keys imported in s3. It can then be sorted, find files after or before a date, matching a date ...

Example :  To extract all files for a given date

DATE=$(date +%Y-%m-%d) aws s3api list-objects-v2 --bucket test-bucket-fh --query 'Contents[?contains(LastModified, `$DATE`)]'

s3api will return a matadata where you can filter for specific elements by command 

DATE=$(date +%Y-%m-%d)
aws s3api list-objects-v2 --bucket test-bucket-fh --query 'Contents[?contains(LastModified, `$DATE`)].Key'

To learn more, refer to the AWS Course in Bangalore.

Comments

Getting NUll list

---

How to use this s3api to download files from s3 bucket?? could you please explain that with an example?

I am currently using following command to download files form s3 but I need to download them based on month, so any help would be appreciated!

aws --endpoint-url https://example.com s3 cp s3://objects/EOB/  . --recursive

---

Hey @Vikram, you can use the --include and --exclude flags in the command. Something like this:

aws s3 cp s3://bucket/ folder --exclude "*" --include "2019-07-01*" --recursive

This code downloads all the files starting from 2019-07-01