Read file content from S3 bucket with boto3

+1 vote

I read the filenames in my S3 bucket by doing

objs = boto3.client.list_objects(Bucket='my_bucket') while 'Contents' in objs.keys(): objs_contents = objs['Contents'] for i in range(len(objs_contents)): filename = objs_contents[i]['Key']

Now, I need to get the actual content of the file, similarly to a open(filename).readlines(). What is the best way?

Oct 23, 2018 in AWS by datageek
• 2,440 points

2 answers to this question.

+1 vote

boto3 offers a resource model that makes tasks like iterating through objects easier. Unfortunately, StreamingBody doesn't provide readline or readlines.

s3 = boto3.resource('s3')
bucket = s3.Bucket('test-bucket')
# Iterates through all the objects, doing the pagination for you. Each obj
# is an ObjectSummary, so it doesn't contain the body. You'll need to call
# get to get the whole body.
for obj in bucket.objects.all():
    key = obj.key
    body = obj.get()['Body'].read()
answered Oct 23, 2018 by anonymous
+1 vote
bucket = s3_client.Bucket('test')
for obj in bucket.objects.all():
for line in contents.splitlines():
answered Jul 3 by reddy

edited Jul 4 by Kalgi

Related Questions In AWS

0 votes
0 answers

I want to get file name from key in S3 bucket wanted to read single file from list of file present in bucket

1 <class 'boto.s3.key.Key'> <Key: numbers-email, staging/Procured_Numbers_Status/procured_numbers_status_2019-05-15:06:09:04.csv> I ...READ MORE

May 15 in AWS by anonymous
+1 vote
2 answers

Want my AWS s3 Bucket to read Name from CloudWatch Event

CloudTrail events for S3 bucket level operations ...READ MORE

answered May 28, 2018 in AWS by Cloud gunner
• 4,280 points
0 votes
1 answer
+1 vote
11 answers

Python AWS Boto3: How do i read files from S3 Bucket?

All of the answers are kind of ...READ MORE

answered Mar 30 in AWS by awsdbaexpert
• 200 points
0 votes
1 answer

How to copy .csv file from Amazon S3 bucket?

Boto3 is the library to use for ...READ MORE

answered Jul 6, 2018 in AWS by Priyaj
• 57,300 points