Copying folders inside Amazon S3 bucket

0 votes

I want to copy one folder, with all existing files inside it, to another folder inside AmazonS3's same Bucket.

I can copy one object but what I want to do is copy the folder, with all files, into another Folder.

Nov 29, 2018 in AWS by datageek
• 3,040 points
1,009 views

2 answers to this question.

0 votes

Hey, this works for me.  

  public bool CopyFolderInsideS3Bucket(string source, string destination)
        {
            var strippedSource = source;
            var strippedDestination = destination;

            // process source
            if (strippedSource.StartsWith("/"))
                strippedSource = strippedSource.Substring(1);
            if (strippedSource.EndsWith("/"))
                strippedSource = source.Substring(0, strippedSource.Length - 1);

            var sourceParts = strippedSource.Split('/');
            var sourceBucket = sourceParts[0];

            var sourcePrefix = new StringBuilder();
            for (var i = 1; i < sourceParts.Length; i++)
            {
                sourcePrefix.Append(sourceParts[i]);
                sourcePrefix.Append("/");
            }

            // process destination
            if (strippedDestination.StartsWith("/"))
                strippedDestination = destination.Substring(1);
            if (strippedDestination.EndsWith("/"))
                strippedDestination = destination.Substring(0, strippedDestination.Length - 1);

            var destinationParts = strippedDestination.Split('/');
            var destinationBucket = destinationParts[0];

            var destinationPrefix = new StringBuilder();
            for (var i = 1; i < destinationParts.Length; i++)
            {
                destinationPrefix.Append(destinationParts[i]);
                destinationPrefix.Append("/");
            }

            var listObjectsResult = client.ListObjects(new ListObjectsRequest(){ 
                BucketName = sourceBucket,
                Prefix = sourcePrefix.ToString(),
                Delimiter = "/"});

            // copy each file
            foreach (var file in listObjectsResult.S3Objects)
            {
                var request = new CopyObjectRequest();
                request.SourceBucket = Settings.BucketName;
                request.SourceKey = file.Key;
                request.DestinationBucket = destinationBucket;
                request.DestinationKey = destinationPrefix + file.Key.Substring(sourcePrefix.Length);
                request.CannedACL = S3CannedACL.PublicRead;
                var response = (CopyObjectResponse)client.CopyObject(request);
            }

            // copy subfolders
            foreach (var folder in listObjectsResult.CommonPrefixes)
            {
                var actualFolder = folder.Substring(sourcePrefix.Length);
                actualFolder = actualFolder.Substring(0, actualFolder.Length - 1);
                CopyFolderInsideS3Bucket(strippedSource + "/" + actualFolder, strippedDestination + "/" + actualFolder);
            }

            return true;
        }

Hope this helped!

answered Nov 29, 2018 by anonymous
Yes this also worked fine. Thank you!
0 votes

Here is the example to copy folder inside AmazonS3 Bucket which works for me

public bool CopyFolderInsideS3Bucket(string source, string destination)
        {
            var strippedSource = source;
            var strippedDestination = destination;

            // process source
            if (strippedSource.StartsWith("/"))
                strippedSource = strippedSource.Substring(1);
            if (strippedSource.EndsWith("/"))
                strippedSource = source.Substring(0, strippedSource.Length - 1);

            var sourceParts = strippedSource.Split('/');
            var sourceBucket = sourceParts[0];

            var sourcePrefix = new StringBuilder();
            for (var i = 1; i < sourceParts.Length; i++)
            {
                sourcePrefix.Append(sourceParts[i]);
                sourcePrefix.Append("/");
            }

            // process destination
            if (strippedDestination.StartsWith("/"))
                strippedDestination = destination.Substring(1);
            if (strippedDestination.EndsWith("/"))
                strippedDestination = destination.Substring(0, strippedDestination.Length - 1);

            var destinationParts = strippedDestination.Split('/');
            var destinationBucket = destinationParts[0];

            var destinationPrefix = new StringBuilder();
            for (var i = 1; i < destinationParts.Length; i++)
            {
                destinationPrefix.Append(destinationParts[i]);
                destinationPrefix.Append("/");
            }
Hope this helped!
answered Nov 29, 2018 by Archana
• 5,560 points

Related Questions In AWS

0 votes
1 answer

How do I create folder under an Amazon S3 bucket through PHP API?

Of Course, it is possible to create ...READ MORE

answered Apr 24, 2018 in AWS by anonymous
2,270 views
0 votes
1 answer

How to copy .csv file from Amazon S3 bucket?

Boto3 is the library to use for ...READ MORE

answered Jul 6, 2018 in AWS by Priyaj
• 56,520 points
378 views
0 votes
1 answer

Amazon s3 Folders Problem

Amazon S3 is a flat file system. ...READ MORE

answered Jul 13, 2018 in AWS by Cloud gunner
• 4,260 points
44 views
+1 vote
2 answers

AWS CloudWatch Logs in Docker

The awslogs works without using ECS. you need to configure ...READ MORE

answered Sep 6, 2018 in AWS by bug_seeker
• 15,350 points
326 views
0 votes
1 answer

Deploy Docker Containers from Docker Cloud

To solve this problem, I followed advice ...READ MORE

answered Sep 3, 2018 in AWS by Priyaj
• 56,520 points
149 views
+3 votes
5 answers

How to move files from amazon ec2 to s3 bucket using command line

Hey, 3 ways you can do this: To ...READ MORE

answered Oct 9, 2018 in AWS by Omkar
• 67,380 points
2,497 views
+1 vote
4 answers

How do I cache my images which are stored in Amazon S3?

when caching ec2 instance these can be ...READ MORE

answered Oct 23, 2018 in AWS by Nabarupa
601 views