How to list all the objects in Amazon S3?

0 votes

S3Client.ListObjects return only 1000 of objects. How to retrieve list of all existing objects using Amazon C# library?

Oct 5, 2018 in AWS by datageek
• 2,440 points
187 views

1 answer to this question.

0 votes

As stated already, Amazon S3 indeed requires Listing Keys Using the AWS SDK for .NET:

As buckets can contain a virtually unlimited number of keys, the complete results of a list query can be extremely large. To manage large result sets, Amazon S3 uses pagination to split them into multiple responses. Each list keys response returns a page of up to 1,000 keys with an indicator indicating if the response is truncated. You send a series of list keys requests until you have received all the keys.

The mentioned indicator is the NextMarker property from the ObjectsResponse Class - its usage is illustrated in the complete example Listing Keys Using the AWS SDK for .NET, with the relevant fragment being:

static AmazonS3 client;
client = Amazon.AWSClientFactory.CreateAmazonS3Client(
                    accessKeyID, secretAccessKeyID);

ListObjectsRequest request = new ListObjectsRequest();
request.BucketName = bucketName;
do
{
   ListObjectsResponse response = client.ListObjects(request);

   // Process response.
   // ...

   // If response is truncated, set the marker to get the next 
   // set of keys.
   if (response.IsTruncated)
   {
        request.Marker = response.NextMarker;
   }
   else
   {
        request = null;
   }
} while (request != null);

answered Oct 5, 2018 by Archana
• 4,090 points

Related Questions In AWS

0 votes
1 answer

How to list the contents of Amazon S3 by modified date?

One easy solution would be probably to ...READ MORE

answered Aug 21, 2018 in AWS by datageek
• 2,440 points
3,023 views
+1 vote
1 answer

How to combine multiple S3 objects in the target S3 object w/o leaving S3

"However sometimes our raw objects are not ...READ MORE

answered Aug 28, 2018 in AWS by Priyaj
• 56,160 points
1,393 views
0 votes
1 answer

How to make multiple files in Amazon S3 public?

I had to change several hundred thousand ...READ MORE

answered Oct 17, 2018 in AWS by Archana
• 4,090 points
54 views
+4 votes
5 answers

Usage of Amazon Cloudfront or S3

When to use S3? S3 is like many ...READ MORE

answered Apr 3, 2018 in AWS by brat_1
• 7,080 points
81 views
+3 votes
5 answers

Renaming files in S3

You can either use AWS CLI or ...READ MORE

answered Oct 16, 2018 in AWS by petter dj
600 views
0 votes
1 answer

S3 Static Website Hosting Route All Paths to Index.html

Yes there is an easy way to ...READ MORE

answered Apr 8, 2018 in AWS by code_ninja
• 6,160 points
282 views
+1 vote
4 answers

Can a URL be directly uploaded to S3 using POST?

You can read this blog and get ...READ MORE

answered Oct 25, 2018 in AWS by chamunda
41 views
0 votes
1 answer

How to download the latest file in a S3 bucket using AWS CLI?

You can use the below command $ aws ...READ MORE

answered Sep 6, 2018 in AWS by Archana
• 4,090 points
2,444 views
0 votes
1 answer

How to upload an object into Amazon S3 in Lambda?

I suspect you are calling the context.done() function before s3.upload() has ...READ MORE

answered Sep 27, 2018 in AWS by Archana
• 4,090 points
37 views