How to list all the objects in Amazon S3

0 votes

S3Client.ListObjects return only 1000 of objects. How to retrieve list of all existing objects using Amazon C# library?

Oct 5, 2018 in AWS by datageek
• 2,540 points
7,445 views

1 answer to this question.

0 votes

As stated already, Amazon S3 indeed requires Listing Keys Using the AWS SDK for .NET:

As buckets can contain a virtually unlimited number of keys, the complete results of a list query can be extremely large. To manage large result sets, Amazon S3 uses pagination to split them into multiple responses. Each list keys response returns a page of up to 1,000 keys with an indicator indicating if the response is truncated. You send a series of list keys requests until you have received all the keys.

The mentioned indicator is the NextMarker property from the ObjectsResponse Class - its usage is illustrated in the complete example Listing Keys Using the AWS SDK for .NET, with the relevant fragment being:

static AmazonS3 client;
client = Amazon.AWSClientFactory.CreateAmazonS3Client(
                    accessKeyID, secretAccessKeyID);

ListObjectsRequest request = new ListObjectsRequest();
request.BucketName = bucketName;
do
{
   ListObjectsResponse response = client.ListObjects(request);

   // Process response.
   // ...

   // If response is truncated, set the marker to get the next 
   // set of keys.
   if (response.IsTruncated)
   {
        request.Marker = response.NextMarker;
   }
   else
   {
        request = null;
   }
} while (request != null);

answered Oct 5, 2018 by Archana
• 4,170 points

Related Questions In AWS

0 votes
1 answer

How to list the contents of Amazon S3 by modified date?

One easy solution would be probably to ...READ MORE

answered Aug 21, 2018 in AWS by datageek
• 2,540 points
53,527 views
+2 votes
1 answer

How to combine multiple S3 objects in the target S3 object w/o leaving S3

"However sometimes our raw objects are not ...READ MORE

answered Aug 28, 2018 in AWS by Priyaj
• 58,100 points
12,913 views
0 votes
1 answer

How to list down all the distributions in Cloudfront?

Hi@akhtar, You can list down all the distributions ...READ MORE

answered Nov 2, 2020 in AWS by MD
• 95,460 points
1,353 views
+4 votes
5 answers

Usage of Amazon Cloudfront or S3

When to use S3? S3 is like many ...READ MORE

answered Apr 3, 2018 in AWS by brat_1
• 7,200 points
1,940 views
+3 votes
6 answers

Renaming files in S3

You can either use AWS CLI or ...READ MORE

answered Oct 16, 2018 in AWS by petter dj
31,120 views
0 votes
1 answer

S3 Static Website Hosting Route All Paths to Index.html

Yes there is an easy way to ...READ MORE

answered Apr 8, 2018 in AWS by code_ninja
• 6,300 points
3,418 views
+1 vote
4 answers

Can a URL be directly uploaded to S3 using POST?

You can read this blog and get ...READ MORE

answered Oct 25, 2018 in AWS by chamunda
3,247 views
0 votes
1 answer

How to download the latest file in a S3 bucket using AWS CLI?

You can use the below command $ aws ...READ MORE

answered Sep 6, 2018 in AWS by Archana
• 4,170 points
19,816 views
0 votes
1 answer

How to upload an object into Amazon S3 in Lambda?

I suspect you are calling the context.done() function before s3.upload() has ...READ MORE

answered Sep 27, 2018 in AWS by Archana
• 4,170 points
1,170 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP