Is it possible to perform a batch upload to Amazon S3

0 votes

Is it possible to perform a batch upload to Amazon S3? I have a project that needs to upload each night ~100K of files that can be up to 1G but most of them are files of small sizes. 

Does the s3 API support uploading multiple objects in a single HTTP call?

All the objects must be available in S3 as individual objects. I cannot host them anywhere else. That is an external requirement that I cannot change.

Feb 27, 2019 in AWS by datageek
• 3,090 points
2,899 views

1 answer to this question.

0 votes

You can upload S3 via AWS CLI tool using the sync command.

aws s3 sync local_folder s3://bucket-name

You can use this method to batch upload files to S3 very fast.

answered Feb 27, 2019 by Archana
• 5,640 points

Related Questions In AWS

0 votes
1 answer

How to upload a file to Amazon S3 without passing it my server?

This article pretty much explains the entire ...READ MORE

answered Aug 14, 2018 in AWS by Archana
• 4,170 points
3,192 views
0 votes
1 answer

Is it possible to find all S3 buckets given a prefix?

The high level collection command s3.buckets.filter only ...READ MORE

answered Aug 27, 2018 in AWS by Archana
• 4,170 points
6,546 views
0 votes
1 answer

Is it possible to use own scheduler with Amazon ECS?

ECS provides Blox, a collection of open ...READ MORE

answered Nov 5, 2018 in AWS by datageek
• 3,090 points
708 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

Is it possible to add a compatibility layer above AWS?

Well there are few tools like TyphoonAE ...READ MORE

answered Jan 15, 2019 in AWS by Archana
• 5,640 points
559 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP