Is it possible to perform a batch upload to Amazon S3?

0 votes

Is it possible to perform a batch upload to Amazon S3? I have a project that needs to upload each night ~100K of files that can be up to 1G but most of them are files of small sizes. 

Does the s3 API support uploading multiple objects in a single HTTP call?

All the objects must be available in S3 as individual objects. I cannot host them anywhere else. That is an external requirement that I cannot change.

Feb 26 in AWS by datageek
• 2,830 points
14 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

You can upload S3 via AWS CLI tool using the sync command.

aws s3 sync local_folder s3://bucket-name

You can use this method to batch upload files to S3 very fast.

answered Feb 26 by Archana
• 5,280 points

Related Questions In AWS

0 votes
1 answer

How to upload a file to Amazon S3 without passing it my server?

This article pretty much explains the entire ...READ MORE

answered Aug 14, 2018 in AWS by Archana
• 3,770 points
34 views
0 votes
1 answer

Is it possible to find all S3 buckets given a prefix?

The high level collection command s3.buckets.filter only ...READ MORE

answered Aug 27, 2018 in AWS by Archana
• 3,770 points
141 views
0 votes
1 answer

Is it possible to use own scheduler with Amazon ECS?

ECS provides Blox, a collection of open ...READ MORE

answered Nov 5, 2018 in AWS by datageek
• 2,830 points
18 views
0 votes
1 answer

Is it possible to add a compatibility layer above AWS?

Well there are few tools like TyphoonAE ...READ MORE

answered Jan 14 in AWS by Archana
• 5,280 points
12 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.