How to list all the objects in Amazon S3

0 votes

S3Client.ListObjects return only 1000 of objects. How to retrieve list of all existing objects using Amazon C# library?

Oct 5, 2018 in AWS by datageek
• 2,530 points

1 answer to this question.

0 votes

As stated already, Amazon S3 indeed requires Listing Keys Using the AWS SDK for .NET:

As buckets can contain a virtually unlimited number of keys, the complete results of a list query can be extremely large. To manage large result sets, Amazon S3 uses pagination to split them into multiple responses. Each list keys response returns a page of up to 1,000 keys with an indicator indicating if the response is truncated. You send a series of list keys requests until you have received all the keys.

The mentioned indicator is the NextMarker property from the ObjectsResponse Class - its usage is illustrated in the complete example Listing Keys Using the AWS SDK for .NET, with the relevant fragment being:

static AmazonS3 client;
client = Amazon.AWSClientFactory.CreateAmazonS3Client(
                    accessKeyID, secretAccessKeyID);

ListObjectsRequest request = new ListObjectsRequest();
request.BucketName = bucketName;
   ListObjectsResponse response = client.ListObjects(request);

   // Process response.
   // ...

   // If response is truncated, set the marker to get the next 
   // set of keys.
   if (response.IsTruncated)
        request.Marker = response.NextMarker;
        request = null;
} while (request != null);

answered Oct 5, 2018 by Archana
• 4,150 points

Related Questions In AWS

0 votes
1 answer

How to list the contents of Amazon S3 by modified date?

One easy solution would be probably to ...READ MORE

answered Aug 21, 2018 in AWS by datageek
• 2,530 points
+2 votes
1 answer

How to combine multiple S3 objects in the target S3 object w/o leaving S3

"However sometimes our raw objects are not ...READ MORE

answered Aug 28, 2018 in AWS by Priyaj
• 58,140 points
0 votes
1 answer

How to list down all the distributions in Cloudfront?

Hi@akhtar, You can list down all the distributions ...READ MORE

answered Nov 2, 2020 in AWS by MD
• 95,300 points
+4 votes
5 answers

Usage of Amazon Cloudfront or S3

When to use S3? S3 is like many ...READ MORE

answered Apr 3, 2018 in AWS by brat_1
• 7,200 points
+3 votes
6 answers

Renaming files in S3

You can either use AWS CLI or ...READ MORE

answered Oct 16, 2018 in AWS by petter dj
0 votes
1 answer

S3 Static Website Hosting Route All Paths to Index.html

Yes there is an easy way to ...READ MORE

answered Apr 8, 2018 in AWS by code_ninja
• 6,280 points
+1 vote
4 answers

Can a URL be directly uploaded to S3 using POST?

You can read this blog and get ...READ MORE

answered Oct 25, 2018 in AWS by chamunda
0 votes
1 answer

How to download the latest file in a S3 bucket using AWS CLI?

You can use the below command $ aws ...READ MORE

answered Sep 6, 2018 in AWS by Archana
• 4,150 points
0 votes
1 answer

How to upload an object into Amazon S3 in Lambda?

I suspect you are calling the context.done() function before s3.upload() has ...READ MORE

answered Sep 27, 2018 in AWS by Archana
• 4,150 points