How to access files in S3 bucket from R

0 votes

I have installed R on one of my Amazon EC2 Instance. And I have uploaded few csv files into a S3 bucket. I was wondering if there is a way to access these csv files in the S3 bucket from R.

Any help  would be appreciated.

Aug 10, 2018 in AWS by datageek
• 2,540 points
6,596 views

2 answers to this question.

0 votes

Enter the following command: install.packages("AWS.tools")

Then, use the s3.get() command from there. The Help tab should tell you what goes in for arguments.

answered Aug 10, 2018 by Archana
• 4,170 points
0 votes
You can take a look at the cloudyr aws.s3 package in Github ( https://github.com/cloudyr/aws.s3 ), it is quite related to what you need. Unfortunately, this package is quite early-stage & a little unstable.

I've have had good success simply using R's system() command to make a call to the AWS CLI. This is relatively easy to get started on, very robust, and very well supported.

Start here: https://aws.amazon.com/cli/

List objects using S3 API: https://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects.html

Get objects using S3 API: https://docs.aws.amazon.com/cli/latest/reference/s3api/get-object.html

So, for example, on command-line try following:

pip install awscli aws configure aws s3 help aws s3api list-objects --bucket some-bucket --query 'Contents[].{Key: Key}' aws s3api get-object --bucket some-bucket --key some_file.csv new_file_name.csv

In R, can just do something like:

system("aws s3api list-objects --bucket some-bucket --query 'Contents[].{Key: Key}' > my_bucket.json")
answered Aug 10, 2018 by Deepthi
• 300 points

Related Questions In AWS

0 votes
1 answer

How to edit files in the AWS S3 bucket from CLI?

Hi@akhtar, According to my knowledge,  you can't edit ...READ MORE

answered Jun 8, 2020 in AWS by MD
• 95,460 points
9,637 views
+3 votes
6 answers

How to move files from amazon ec2 to s3 bucket using command line

Hey, 3 ways you can do this: To ...READ MORE

answered Oct 9, 2018 in AWS by Omkar
• 69,220 points
19,948 views
0 votes
1 answer

how to access AWS S3 from Lambda in VPC

With boto3, the S3 urls are virtual by default, ...READ MORE

answered Sep 28, 2018 in AWS by Priyaj
• 58,100 points
10,089 views
0 votes
1 answer

How much will it cost to read from an S3 bucket in the same region but on different account

There will be no data transfer cost ...READ MORE

answered Feb 17, 2022 in AWS by anonymous
3,114 views
+6 votes
14 answers

Python AWS Boto3: How do i read files from S3 Bucket?

You can use the following code, import boto3 s3 ...READ MORE

answered Dec 7, 2018 in AWS by Nitesh
315,424 views
0 votes
1 answer

How to download the latest file in a S3 bucket using AWS CLI?

You can use the below command $ aws ...READ MORE

answered Sep 6, 2018 in AWS by Archana
• 4,170 points
19,810 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP