How do you concatenate small sized files in Amazon S3

0 votes

Is it possible to concatenate small files which are less than 5 MBs on Amazon S3? Multi-Part Upload is not suitable for small files.It's not a efficient solution to pull down all these files and do the concatenation.

So, are there any APIs to concatenate files in AWS?

Sep 3, 2018 in AWS by datageek
• 2,530 points

1 answer to this question.

0 votes

Amazon S3 does not provide a concatenate function. It is primarily an object storage service.

You will need some process that downloads the objects, combines them, then uploads them again. The most efficient way to do this would be to download the objects in parallel, to take full advantage of available bandwidth. However, that is more complex to code.

Doing the processing on Amazon EC2 or AWS Lambda would be more efficient and less costly.

answered Sep 3, 2018 by Archana
• 4,150 points

Related Questions In AWS

+1 vote
4 answers

How do I cache my images which are stored in Amazon S3?

when caching ec2 instance these can be ...READ MORE

answered Oct 23, 2018 in AWS by Nabarupa
0 votes
0 answers

how can i see / access my files in amazon s3

I need to know how can I ...READ MORE

Mar 7, 2020 in AWS by Prince
• 120 points
0 votes
1 answer

How do I create folder under an Amazon S3 bucket through PHP API?

Of Course, it is possible to create ...READ MORE

answered Apr 24, 2018 in AWS by anonymous
+3 votes
6 answers

How to move files from amazon ec2 to s3 bucket using command line

Hey, 3 ways you can do this: To ...READ MORE

answered Oct 9, 2018 in AWS by Omkar
• 69,150 points
+4 votes
5 answers

Usage of Amazon Cloudfront or S3

When to use S3? S3 is like many ...READ MORE

answered Apr 3, 2018 in AWS by brat_1
• 7,180 points
+3 votes
6 answers

Renaming files in S3

You can either use AWS CLI or ...READ MORE

answered Oct 16, 2018 in AWS by petter dj
0 votes
1 answer

S3 Static Website Hosting Route All Paths to Index.html

Yes there is an easy way to ...READ MORE

answered Apr 8, 2018 in AWS by code_ninja
• 6,280 points
+1 vote
4 answers

Can a URL be directly uploaded to S3 using POST?

You can read this blog and get ...READ MORE

answered Oct 25, 2018 in AWS by chamunda
0 votes
1 answer

How to make multiple files in Amazon S3 public?

I had to change several hundred thousand ...READ MORE

answered Oct 17, 2018 in AWS by Archana
• 4,150 points
0 votes
2 answers

How to access files in S3 bucket from R?

You can take a look at the ...READ MORE

answered Aug 10, 2018 in AWS by Deepthi
• 300 points