How do you concatenate small sized files in Amazon S3?

0 votes

Is it possible to concatenate small files which are less than 5 MBs on Amazon S3? Multi-Part Upload is not suitable for small files.It's not a efficient solution to pull down all these files and do the concatenation.

So, are there any APIs to concatenate files in AWS?

Sep 3, 2018 in AWS by datageek
• 2,460 points
1,300 views

1 answer to this question.

0 votes

Amazon S3 does not provide a concatenate function. It is primarily an object storage service.

You will need some process that downloads the objects, combines them, then uploads them again. The most efficient way to do this would be to download the objects in parallel, to take full advantage of available bandwidth. However, that is more complex to code.

Doing the processing on Amazon EC2 or AWS Lambda would be more efficient and less costly.

answered Sep 3, 2018 by Archana
• 4,150 points

Related Questions In AWS

+1 vote
4 answers

How do I cache my images which are stored in Amazon S3?

when caching ec2 instance these can be ...READ MORE

answered Oct 23, 2018 in AWS by Nabarupa
1,407 views
0 votes
1 answer

How do I create folder under an Amazon S3 bucket through PHP API?

Of Course, it is possible to create ...READ MORE

answered Apr 24, 2018 in AWS by anonymous
3,409 views
+3 votes
5 answers

How to move files from amazon ec2 to s3 bucket using command line

Hey, 3 ways you can do this: To ...READ MORE

answered Oct 9, 2018 in AWS by Omkar
• 68,880 points
5,378 views
+2 votes
11 answers

Python AWS Boto3: How do i read files from S3 Bucket?

You can download the file from S3 ...READ MORE

answered Dec 7, 2018 in AWS by Jino
69,360 views
+4 votes
5 answers

Usage of Amazon Cloudfront or S3

When to use S3? S3 is like many ...READ MORE

answered Apr 3, 2018 in AWS by brat_1
• 7,160 points
156 views
+3 votes
5 answers

Renaming files in S3

You can either use AWS CLI or ...READ MORE

answered Oct 16, 2018 in AWS by petter dj
1,687 views
0 votes
1 answer

S3 Static Website Hosting Route All Paths to Index.html

Yes there is an easy way to ...READ MORE

answered Apr 8, 2018 in AWS by code_ninja
• 6,230 points
771 views
+1 vote
4 answers

Can a URL be directly uploaded to S3 using POST?

You can read this blog and get ...READ MORE

answered Oct 25, 2018 in AWS by chamunda
109 views
0 votes
1 answer

How to make multiple files in Amazon S3 public?

I had to change several hundred thousand ...READ MORE

answered Oct 17, 2018 in AWS by Archana
• 4,150 points
89 views
0 votes
2 answers

How to access files in S3 bucket from R?

You can take a look at the ...READ MORE

answered Aug 10, 2018 in AWS by Deepthi
• 300 points
1,759 views