Is it possible to perform a batch upload to Amazon S3?

0 votes

Is it possible to perform a batch upload to Amazon S3? I have a project that needs to upload each night ~100K of files that can be up to 1G but most of them are files of small sizes. 

Does the s3 API support uploading multiple objects in a single HTTP call?

All the objects must be available in S3 as individual objects. I cannot host them anywhere else. That is an external requirement that I cannot change.

Feb 26 in AWS by datageek
• 3,040 points
25 views

1 answer to this question.

0 votes

You can upload S3 via AWS CLI tool using the sync command.

aws s3 sync local_folder s3://bucket-name

You can use this method to batch upload files to S3 very fast.

answered Feb 26 by Archana
• 5,560 points

Related Questions In AWS

0 votes
1 answer

How to upload a file to Amazon S3 without passing it my server?

This article pretty much explains the entire ...READ MORE

answered Aug 14, 2018 in AWS by Archana
• 4,090 points
66 views
0 votes
1 answer

Is it possible to find all S3 buckets given a prefix?

The high level collection command s3.buckets.filter only ...READ MORE

answered Aug 27, 2018 in AWS by Archana
• 4,090 points
490 views
0 votes
1 answer

Is it possible to use own scheduler with Amazon ECS?

ECS provides Blox, a collection of open ...READ MORE

answered Nov 5, 2018 in AWS by datageek
• 3,040 points
27 views
0 votes
1 answer

Is it possible to add a compatibility layer above AWS?

Well there are few tools like TyphoonAE ...READ MORE

answered Jan 14 in AWS by Archana
• 5,560 points
18 views