What is the algorithm to compute the Amazon-S3 Etag for a file larger than 5GB

0 votes

Files uploaded to Amazon S3 that are smaller than 5GB have an ETag that is simply the MD5 hash of the file, which makes it easy to check if your local files are the same as what you put on S3.

But if your file is larger than 5GB, then Amazon computes the ETag differently.

For example, I did a multipart upload of a 5,970,150,664 byte file in 380 parts. Now S3 shows it to have an ETag of 6bcf86bed8807b8e78f0fc6e0a53079d-380. My local file has an md5 hash of 702242d3703818ddefe6bf7da2bed757. I think the number after the dash is the number of parts in the multipart upload.

I also suspect that the new ETag is still an MD5 hash, but with some metadata included along the way from the multipart upload somehow.

Does anyone know how to compute the ETag using the same algorithm as Amazon S3? Any help will be appreciated!!

Apr 21 in AWS by Soham
• 8,730 points
24 views

No answer to this question. Be the first to respond.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.

Related Questions In AWS

0 votes
1 answer
0 votes
1 answer

What is the difference between Amazon Cloudfront and S3

major difference is that cloudfront allows you ...READ MORE

answered Jul 10, 2018 in AWS by Priyaj
• 58,060 points
1,150 views
0 votes
1 answer

How to upload a file to Amazon S3 without passing it my server?

This article pretty much explains the entire ...READ MORE

answered Aug 14, 2018 in AWS by Archana
• 4,150 points
2,196 views
0 votes
1 answer

How to download the latest file in a S3 bucket using AWS CLI?

You can use the below command $ aws ...READ MORE

answered Sep 6, 2018 in AWS by Archana
• 4,150 points
14,399 views
0 votes
1 answer

AWS S3 uploading hidden files by default

versioning is enabled in your bucket. docs.aws.amazon.com/AmazonS3/latest/user-guide/….... the ...READ MORE

answered Oct 4, 2018 in AWS by Priyaj
• 58,060 points
3,165 views
–1 vote
1 answer

How to decrypt the encrypted S3 file using aws-encryption-cli --decrypt

Use command : aws s3 presign s3://mybucket/abc_count.png you get ...READ MORE

answered Oct 22, 2018 in AWS by Priyaj
• 58,060 points
3,473 views
0 votes
1 answer

Import my AWS credentials using python script

Using AWS Cli  Configure your IAM user then ...READ MORE

answered Nov 16, 2018 in AWS by Jino
• 5,800 points
1,870 views
0 votes
2 answers
webinar REGISTER FOR FREE WEBINAR X
Send OTP
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP