How to access files in S3 bucket from R?

0 votes

I have installed R on one of my Amazon EC2 Instance. And I have uploaded few csv files into a S3 bucket. I was wondering if there is a way to access these csv files in the S3 bucket from R.

Any help  would be appreciated.

Aug 10, 2018 in AWS by datageek
• 2,430 points
480 views

2 answers to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

Enter the following command: install.packages("AWS.tools")

Then, use the s3.get() command from there. The Help tab should tell you what goes in for arguments.

answered Aug 10, 2018 by Archana
• 4,090 points
0 votes

You can take a look at the cloudyr aws.s3 package (https://github.com/cloudyr/aws.s3), it is quite related to what you need. Unfortunately , this package is quite early stage & a little unstable.

I've have had good success simply using R's system() command to make a call to the AWS CLI. This is relatively easy to get started on, very robust and very well supported.

  1. Start here: http://aws.amazon.com/cli/

  2. List objects using S3 API: http://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects.html

  3. Get objects using S3 API: http://docs.aws.amazon.com/cli/latest/reference/s3api/get-object.html

So, for example, on command-line try following:

pip install awscli aws configure aws s3 help aws s3api list-objects --bucket some-bucket --query 'Contents[].{Key: Key}' aws s3api get-object --bucket some-bucket --key some_file.csv new_file_name.csv

In R, can just do something like:

system("aws s3api list-objects --bucket some-bucket --query 'Contents[].{Key: Key}' > my_bucket.json")

answered Aug 10, 2018 by Deepthi
• 300 points

Related Questions In AWS

+3 votes
5 answers

How to move files from amazon ec2 to s3 bucket using command line

First Option sm3cmd Use s3cmd s3cmd get s3://AWS_S3_Bucket/dir/file Take a ...READ MORE

answered Aug 29, 2018 in AWS by Priyaj
• 56,140 points
1,069 views
0 votes
1 answer

how to access AWS S3 from Lambda in VPC

With boto3, the S3 urls are virtual by default, ...READ MORE

answered Sep 28, 2018 in AWS by Priyaj
• 56,140 points
1,032 views
0 votes
1 answer

How to copy .csv file from Amazon S3 bucket?

Boto3 is the library to use for ...READ MORE

answered Jul 6, 2018 in AWS by Priyaj
• 56,140 points
296 views
0 votes
1 answer
0 votes
1 answer

Copying existing files in a s3 Bucket to another s3 bucket

Suppose the two buckets in question are ...READ MORE

answered Sep 14, 2018 in AWS by Priyaj
• 56,140 points
38 views
0 votes
1 answer

How to appoint a sub-domain in a S3 bucket?

You need to rename your bucket to ...READ MORE

answered Oct 12, 2018 in AWS by Archana
• 4,090 points
25 views
0 votes
1 answer

How to make multiple files in Amazon S3 public?

I had to change several hundred thousand ...READ MORE

answered Oct 17, 2018 in AWS by Archana
• 4,090 points
47 views
+1 vote
11 answers

Python AWS Boto3: How do i read files from S3 Bucket?

All of the answers are kind of ...READ MORE

answered Mar 30 in AWS by awsdbaexpert
• 200 points
17,082 views
0 votes
1 answer

How to download the latest file in a S3 bucket using AWS CLI?

You can use the below command $ aws ...READ MORE

answered Sep 6, 2018 in AWS by Archana
• 4,090 points
1,918 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.