How do I read a csv stored in S3 with csv.DictReader?

0 votes

So, I have code that fetches an AWS S3 object. How do I read this StreamingBody with Python's csv.DictReader?

import boto3, csv

session = boto3.session.Session(aws_access_key_id=<>, aws_secret_access_key=<>, region_name=<>)
s3_resource = session.resource('s3')
s3_object = s3_resource.Object(<bucket>, <key>)
streaming_body = s3_object.get()['Body']

#csv.DictReader(.............)?
So, how do i do that?
Nov 27, 2018 in AWS by datageek
• 3,000 points
507 views

1 answer to this question.

0 votes

Hey, I have attached code line by line. The code would be something like this:

import boto3
import csv

# get a handle on s3
s3 = boto3.resource(u's3')

# get a handle on the bucket that holds your file
bucket = s3.Bucket(u'bucket-name')

# get a handle on the object you want (i.e. your file)
obj = bucket.Object(key=u'test.csv')

# get the object
response = obj.get()

# read the contents of the file and split it into a list of lines

lines = response[u'Body'].read().split()

# now iterate over those lines
for row in csv.DictReader(lines):

    # here you get a sequence of dicts
    # do whatever you want with each line here
    print(row)

You can compact this a bit in actual code, but I tried to keep it step-by-step to show the object hierarchy with boto3.

answered Nov 27, 2018 by Archana
• 5,480 points

Related Questions In AWS

+1 vote
4 answers

How do I cache my images which are stored in Amazon S3?

when caching ec2 instance these can be ...READ MORE

answered Oct 23, 2018 in AWS by Nabarupa
390 views
0 votes
1 answer

How do I write an S3 Object to a file?

While IOUtils.copy() and IOUtils.copyLarge() are great, I would prefer the old ...READ MORE

answered Jul 13, 2018 in AWS by Hammer
• 360 points
67 views
+1 vote
11 answers

Python AWS Boto3: How do i read files from S3 Bucket?

All of the answers are kind of ...READ MORE

answered Mar 30 in AWS by awsdbaexpert
• 200 points
22,076 views
0 votes
1 answer

How do I disable detailed monitoring for instances in an auto scaling group in a CloudFormation template?

The property you want is InstanceMonitoring, not ...READ MORE

answered Aug 31, 2018 in AWS by Archana
• 4,090 points
35 views
+1 vote
2 answers

AWS CloudWatch Logs in Docker

The awslogs works without using ECS. you need to configure ...READ MORE

answered Sep 6, 2018 in AWS by bug_seeker
• 15,310 points
277 views
0 votes
1 answer
–1 vote
1 answer

How to read a csv file stored in Amazon S3 using csv.DictReader

The code would be something like this: import ...READ MORE

answered Oct 25, 2018 in AWS by Archana
• 5,480 points
5,340 views