How do I cache my images which are stored in Amazon S3?

+1 vote
I have some of my web services running on Amazon EC2. As my application needs to deal with a large number of photos, I plan to board them on my Amazon S3. Is there any way to cache the images on EC2? I'm using the EC2 micro instance and I feel it'd be better if I do not cache the Image on EC2. But the scale might grow fast and eventually will need an image cache. If a cache is needed, is it better to do it on EC2, or on S3? (Is there a way for caching for S3. Also, when the client uploads an image should it be uploaded to EC2 or S3 directly?
Apr 11, 2018 in AWS by Cloudgunner
379 views

4 answers to this question.

+1 vote

Why use EC2 here? 

I'd strongly suggest you should use CloudFront for this case as when you use CloudFront with S3 as an origin, your content will get distributed to 49 different locations worldwide directly working out as a cache globally and the content which is being fetched from the nearest location based on the latency to the end users.You don't need to worry about the scale and performance of Cache, EC2 can straightforward offload this to CloudFront and S3.

However, if you're  very keen to work on EC2 only then I am sharing few scenarios for EC2 caching instance:

  • your upload/download ratio is far from 50/50
  • you hit S3 limit 100req/sec
  • you need URL Masking
  • you want to optimize kernel, TCP/IP settings, cache SSL session for clients
  • you want proper cache invalidating mechanism for all geo locations
  • you need 100% control where data is stored
  • you need to count the number of requests
  • you have the custom authentication mechanism

answered Apr 11, 2018 by Flying geek
• 3,150 points
+1 vote

proper use of the Cache-Control header will make life better. With that header you can tell a browser how long the content is good for. If it is something that never changes, you can instruct a browser to keep it for a year. If it frequently changes, you can instruct a browser to keep it for an hour, or a minute, or revalidate every time. You can give similar instructions to a CDN.

Here's a good guide, and here are some examples:

# keep for one year
Cache-Control: max-age=2592000

# keep for a day on a CDN, but a minute on client browsers
Cache-Control: s-maxage=86400, maxage=60

You can add this to pages served from your EC2 instance (no matter if it's nginx, Tornado, Tomcat, IIS), you can add it to the headers on S3 files, and CloudFront will use these values.

answered Oct 23, 2018 by abc
+1 vote

When you use CloudFront in conjunction with S3 as origin; the content gets distributed to 49 different locations worldwide ( as of count of edge locations worldwide today ) directly working out as a cache globally and the content being fetched from nearest location based on the latency to your end users.

answered Oct 23, 2018 by kristan
+1 vote

when caching ec2 instance these can be the scenario:

  • your upload/download ratio is far from 50/50

  • you hit S3 limit 100req/sec

  • you need URL masking

  • you want to optimise kernel, TCP/IP settings, cache SSL session for clients

  • you want proper cache invalidating mechanism for all geo locations

  • you need 100% control where data is stored

  • you need to count number of requests

  • you have custom authentication mechanism

answered Oct 23, 2018 by Nabarupa

Related Questions In AWS

0 votes
1 answer

How do I read a csv stored in S3 with csv.DictReader?

Hey, I have attached code line by ...READ MORE

answered Nov 27, 2018 in AWS by Archana
• 5,440 points
498 views
+1 vote
2 answers

How do I get my AWS Glue client in JAVA?

Hey, you've been using a correct code ...READ MORE

answered Apr 17, 2018 in AWS by Cloud gunner
• 4,260 points
759 views
0 votes
1 answer

How do I create folder under an Amazon S3 bucket through PHP API?

Of Course, it is possible to create ...READ MORE

answered Apr 24, 2018 in AWS by anonymous
1,916 views
0 votes
1 answer

How do you concatenate small sized files in Amazon S3?

Amazon S3 does not provide a concatenate ...READ MORE

answered Sep 3, 2018 in AWS by Archana
• 4,090 points
622 views
+13 votes
2 answers
+1 vote
2 answers

AWS CloudWatch Logs in Docker

The awslogs works without using ECS. you need to configure ...READ MORE

answered Sep 6, 2018 in AWS by bug_seeker
• 15,310 points
270 views
0 votes
1 answer
0 votes
2 answers

How do I define the principal for an AWS policy statement?

Check with AWS Policy Documentations once, rest ...READ MORE

answered Jun 25, 2018 in AWS by Cloud gunner
• 4,260 points

edited Jun 26, 2018 by Cloud gunner 68 views