How to list the contents of Amazon S3 by modified date?

0 votes

Most of the time it so happens that we load so many files in a common S3 bucket due to which it becomes hard to figure out data in it.

Is there a way where i can view objects uploaded on a particular date?

Aug 21, 2018 in AWS by Archana
• 4,090 points
5,076 views

1 answer to this question.

0 votes

One easy solution would be probably to use the s3api. It works easily if you have less than 1000 objects, otherwise you might have to use pagination.

s3api lists all objects and has a property for the lastmodified attribute of keys imported in s3. It can then be sorted, find files after or before a date, matching a date ...

Example :  To extract all files for a given date

DATE=$(date +%Y-%m-%d) aws s3api list-objects-v2 --bucket test-bucket-fh --query 'Contents[?contains(LastModified, `$DATE`)]'

s3api will return a matadata where you can filter for specific elements by command 

DATE=$(date +%Y-%m-%d)
aws s3api list-objects-v2 --bucket test-bucket-fh --query 'Contents[?contains(LastModified, `$DATE`)].Key'
answered Aug 21, 2018 by datageek
• 2,440 points
Getting NUll list

How to use this s3api to download files from s3 bucket?? could you please explain that with an example?

I am currently using following command to download files form s3 but I need to download them based on month, so any help would be appreciated!

aws --endpoint-url https://example.com s3 cp s3://objects/EOB/  . --recursive

Hey @Vikram, you can use the --include and --exclude flags in the command. Something like this:

aws s3 cp s3://bucket/ folder --exclude "*" --include "2019-07-01*" --recursive

This code downloads all the files starting from 2019-07-01

Related Questions In AWS

0 votes
1 answer

How to list all the objects in Amazon S3?

As stated already, Amazon S3 indeed requires ...READ MORE

answered Oct 5, 2018 in AWS by Archana
• 4,090 points
424 views
0 votes
2 answers

How to display just the name of files using aws s3 ls command?

aws s3 ls s3://<your_bucket_name>/ | awk '{print ...READ MORE

answered Mar 17 in AWS by anonymous
3,628 views
0 votes
0 answers
0 votes
1 answer
0 votes
1 answer

How to copy .csv file from Amazon S3 bucket?

Boto3 is the library to use for ...READ MORE

answered Jul 6, 2018 in AWS by Priyaj
• 56,900 points
418 views
0 votes
1 answer

How to add SSL certificate to AWS EC2 with the help of new AWS Certificate Manager service

refer this link  https://aws.amazon.com/certificate-manager/faqs/ You can't install the certificates ...READ MORE

answered Jul 19, 2018 in AWS by Priyaj
• 56,900 points
81 views
0 votes
1 answer

How to link AWS Lambda function to Amazon CloudWatch ?

In order to create Log Group and ...READ MORE

answered Jul 20, 2018 in AWS by datageek
• 2,440 points
196 views
0 votes
1 answer

How is Amazon ECS different from AWS Elastic Beanstalk?

AWS Elastic Beanstalk is an application management ...READ MORE

answered Sep 20, 2018 in AWS by datageek
• 2,440 points
68 views