Transfer data using AWS CLI from one s3 bucket to another

0 votes

I am trying to transfer 25 tb of data stored in s3 from one aws account to another s3 bucket in another AWS account (Both where in different regions) using AWS CLI, can anyone suggest me which EC2 instance is better to use and process for data transfer with CLI and mainly how much time it may take to complete the transfer.

Jan 16, 2019 in AWS by Anjali
• 2,950 points
1,292 views
Hi Anjali,

I have a similar situation. Were you able to complete the process? And how much time did it take?

1 answer to this question.

0 votes

Copying is the easy part! Use the AWS Command-Line Interface (CLI):

aws s3 sync s3://source-bucket s3://destination-bucket

Permissions that you will need to check.

What you will need to consider is how to permit access to copy the files. Let's say you have:

  • Account A with Bucket A
  • Account B with Bucket B
  • You wish to copy from Bucket A to Bucket B

You should run the sync command from a user ("User B") in Account B that has permissions to write to Bucket B.

You will also need to add a Bucket Policy to Bucket A that specifically permits access by User B. The policy would look something like:

{
  "Id": "Policy1",
  "Version": "2019-01-15",
  "Statement": [
    {
      "Sid": "ReadOnlyAccess",
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::my-bucket/*",
      "Principal": {
        "AWS": [
          "arn:aws:iam::123456789012:user/user-b"
        ]
      }
    }
  ]
}
answered Jan 16, 2019 by Rishi

Related Questions In AWS

0 votes
1 answer
+1 vote
1 answer
0 votes
0 answers

transfer the ownership of Amazon S3 objects from one AWS account to another

I want to transfer the ownership of ...READ MORE

Oct 9, 2019 in AWS by Hannah
• 18,570 points

closed Oct 9, 2019 by Kalgi 395 views
+1 vote
3 answers

How to get ARN for s3 Bucket using aws cli .

An ARN is a non-opaque, constructible identifier, ...READ MORE

answered Aug 16, 2018 in AWS by Priyaj
• 58,060 points
3,736 views
0 votes
1 answer

AWS S3 uploading hidden files by default

versioning is enabled in your bucket. docs.aws.amazon.com/AmazonS3/latest/user-guide/….... the ...READ MORE

answered Oct 4, 2018 in AWS by Priyaj
• 58,060 points
3,175 views
–1 vote
1 answer

How to decrypt the encrypted S3 file using aws-encryption-cli --decrypt

Use command : aws s3 presign s3://mybucket/abc_count.png you get ...READ MORE

answered Oct 22, 2018 in AWS by Priyaj
• 58,060 points
3,476 views
0 votes
1 answer

Import my AWS credentials using python script

Using AWS Cli  Configure your IAM user then ...READ MORE

answered Nov 16, 2018 in AWS by Jino
• 5,800 points
1,872 views
0 votes
1 answer

AWS cli returns error

Well the most common error would be 1. ...READ MORE

answered Jan 16, 2019 in AWS by vishnu
306 views
webinar REGISTER FOR FREE WEBINAR X
Send OTP
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP