Transfer data using AWS CLI from one s3 bucket to another

0 votes

I am trying to transfer 25 tb of data stored in s3 from one aws account to another s3 bucket in another AWS account (Both where in different regions) using AWS CLI, can anyone suggest me which EC2 instance is better to use and process for data transfer with CLI and mainly how much time it may take to complete the transfer.

Jan 15 in AWS by Anjali
• 2,930 points
187 views
Hi Anjali,

I have a similar situation. Were you able to complete the process? And how much time did it take?

1 answer to this question.

0 votes

Copying is the easy part! Use the AWS Command-Line Interface (CLI):

aws s3 sync s3://source-bucket s3://destination-bucket

Permissions that you will need to check.

What you will need to consider is how to permit access to copy the files. Let's say you have:

  • Account A with Bucket A
  • Account B with Bucket B
  • You wish to copy from Bucket A to Bucket B

You should run the sync command from a user ("User B") in Account B that has permissions to write to Bucket B.

You will also need to add a Bucket Policy to Bucket A that specifically permits access by User B. The policy would look something like:

{
  "Id": "Policy1",
  "Version": "2019-01-15",
  "Statement": [
    {
      "Sid": "ReadOnlyAccess",
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::my-bucket/*",
      "Principal": {
        "AWS": [
          "arn:aws:iam::123456789012:user/user-b"
        ]
      }
    }
  ]
}
answered Jan 15 by Rishi

Related Questions In AWS

0 votes
1 answer
0 votes
0 answers
+1 vote
3 answers

How to get ARN for s3 Bucket using aws cli .

An ARN is a non-opaque, constructible identifier, ...READ MORE

answered Aug 16, 2018 in AWS by Priyaj
• 56,900 points
1,437 views
0 votes
1 answer

AWS S3 uploading hidden files by default

versioning is enabled in your bucket. docs.aws.amazon.com/AmazonS3/latest/user-guide/….... the ...READ MORE

answered Oct 4, 2018 in AWS by Priyaj
• 56,900 points
262 views
0 votes
1 answer

How to decrypt the encrypted S3 file using aws-encryption-cli --decrypt

Use command : aws s3 presign s3://mybucket/abc_count.png you get ...READ MORE

answered Oct 22, 2018 in AWS by Priyaj
• 56,900 points
571 views
0 votes
1 answer

Import my AWS credentials using python script

Using AWS Cli  Configure your IAM user then ...READ MORE

answered Nov 16, 2018 in AWS by Jino
• 5,560 points
377 views
0 votes
1 answer

AWS cli returns error

Well the most common error would be 1. ...READ MORE

answered Jan 15 in AWS by vishnu
48 views