How do I read a csv stored in S3 with csv DictReader

0 votes

So, I have code that fetches an AWS S3 object. How do I read this StreamingBody with Python's csv.DictReader?

import boto3, csv

session = boto3.session.Session(aws_access_key_id=<>, aws_secret_access_key=<>, region_name=<>)
s3_resource = session.resource('s3')
s3_object = s3_resource.Object(<bucket>, <key>)
streaming_body = s3_object.get()['Body']

So, how do i do that?
Nov 27, 2018 in AWS by datageek
• 3,090 points

1 answer to this question.

0 votes

Hey, I have attached code line by line. The code would be something like this:

import boto3
import csv

# get a handle on s3
s3 = boto3.resource(u's3')

# get a handle on the bucket that holds your file
bucket = s3.Bucket(u'bucket-name')

# get a handle on the object you want (i.e. your file)
obj = bucket.Object(key=u'test.csv')

# get the object
response = obj.get()

# read the contents of the file and split it into a list of lines

lines = response[u'Body'].read().split()

# now iterate over those lines
for row in csv.DictReader(lines):

    # here you get a sequence of dicts
    # do whatever you want with each line here

You can compact this a bit in actual code, but I tried to keep it step-by-step to show the object hierarchy with boto3.

answered Nov 27, 2018 by Archana
• 5,640 points

Related Questions In AWS

+1 vote
4 answers

How do I cache my images which are stored in Amazon S3?

when caching ec2 instance these can be ...READ MORE

answered Oct 23, 2018 in AWS by Nabarupa
0 votes
1 answer

How do I write an S3 Object to a file?

While IOUtils.copy() and IOUtils.copyLarge() are great, I would prefer the old ...READ MORE

answered Jul 13, 2018 in AWS by Hammer
• 360 points
+5 votes
14 answers

Python AWS Boto3: How do i read files from S3 Bucket?

You can use the following code, import boto3 s3 ...READ MORE

answered Dec 7, 2018 in AWS by Nitesh
0 votes
1 answer

How do I disable detailed monitoring for instances in an auto scaling group in a CloudFormation template?

The property you want is InstanceMonitoring, not ...READ MORE

answered Aug 31, 2018 in AWS by Archana
• 4,170 points

edited Jun 16, 2023 by Khan Sarfaraz 1,074 views
+1 vote
2 answers

AWS CloudWatch Logs in Docker

The awslogs works without using ECS. you need to configure ...READ MORE

answered Sep 7, 2018 in AWS by bug_seeker
• 15,520 points
0 votes
1 answer
+1 vote
2 answers

How to read a csv file stored in Amazon S3 using csv.DictReader

The code would be something like this: import ...READ MORE

answered Oct 25, 2018 in AWS by Archana
• 5,640 points
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP