Parallel uploads to the same s3 bucket directory with s3cmd

0 votes

I try to run the following code but index.js turns out to be corrupted.

Any idea why?

gzip dist/production/index.js
mv dist/production/index.js.gz dist/production/index.js

s3cmd --access_key="$S3_ACCESS_KEY" --secret_key="$S3_SECRET_KEY" \
      --acl-public --no-mime-magic --progress --recursive         \
      --exclude "dist/production/index.js" \
      put dist/production/ 
      "s3://${BUCKET}/something/${BUILD_IDENTIFIER}/production/" &

s3cmd --access_key="$S3_ACCESS_KEY" --secret_key="$S3_SECRET_KEY" \
      --acl-public --no-mime-magic --progress --recursive         \
      --add-header="Content-Encoding:gzip" \
      put dist/production/index.js 
      "s3://${BUCKET}/something/${BUILD_IDENTIFIER}/production/" &


Notice the & in the end of the two commands that makes two uploads to the same location in parallel.

Sep 5, 2018 in AWS by bug_seeker
• 14,970 points

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

Is your problem is not with the line?

cp dist/production/index.js.gz dist/production/index.js

You are copying gzipped file, not the plain index.js text file.

Hope it helps.


If you are doing it on purpose why not maintain the gz extension. Extensions does lot of things when you handle with browser.

cp dist/production/index.js.gz dist/production/index.js.gz

If you use plain s3 to download and verify the hash, they should be the same file. I did verify it.

answered Sep 5, 2018 by Priyaj
• 56,120 points

Related Questions In AWS

0 votes
1 answer

How to download the latest file in a S3 bucket using AWS CLI?

You can use the below command $ aws ...READ MORE

answered Sep 6, 2018 in AWS by Archana
• 3,770 points
0 votes
1 answer

AWS S3 uploading hidden files by default

versioning is enabled in your bucket.….... the ...READ MORE

answered Oct 4, 2018 in AWS by Priyaj
• 56,120 points
0 votes
1 answer

How to decrypt the encrypted S3 file using aws-encryption-cli --decrypt

Use command : aws s3 presign s3://mybucket/abc_count.png you get ...READ MORE

answered Oct 22, 2018 in AWS by Priyaj
• 56,120 points
0 votes
1 answer

Import my AWS credentials using python script

Using AWS Cli  Configure your IAM user then ...READ MORE

answered Nov 16, 2018 in AWS by Jino
• 5,520 points
0 votes
2 answers
+1 vote
3 answers

How to get ARN for s3 Bucket using aws cli .

An ARN is a non-opaque, constructible identifier, ...READ MORE

answered Aug 16, 2018 in AWS by Priyaj
• 56,120 points

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.