How do you concatenate small sized files in Amazon S3

Question

Is it possible to concatenate small files which are less than 5 MBs on Amazon S3? Multi-Part Upload is not suitable for small files.It's not a efficient solution to pull down all these files and do the concatenation.

So, are there any APIs to concatenate files in AWS?

Answer 1

Amazon S3 does not provide a concatenate function. It is primarily an object storage service.

You will need some process that downloads the objects, combines them, then uploads them again. The most efficient way to do this would be to download the objects in parallel, to take full advantage of available bandwidth. However, that is more complex to code.

Doing the processing on Amazon EC2 or AWS Lambda would be more efficient and less costly.