How to use AWS S3 CLI to dump files to stdout in BASH

0 votes

I'm starting a bash script which will take a path in S3 (as specified to the ls command) and dump the contents of all of the file objects to stdout. Essentially I'd like to replicate cat /path/to/files/* except for S3, e.g. s3cat '/bucket/path/to/files/*'. My first inclination looking at the options is to use the cp command to a temporary file and then cat that.

Has anyone tried this or similar or is there already a command I'm not finding which does it?

Mar 21, 2022 in Others by Edureka
• 13,670 points
1,613 views

1 answer to this question.

0 votes

All file objects must have their contents written to stdout.

To get this, use the aws s3 cp command and provide "-" as the destination. $ aws s3 cp s3:/mybucket/stream.txt is an example of a command.

Is this what you're after? ::

#!/bin/bash

BUCKET=your-bucket-name

for key in `aws s3api list-objects --bucket $BUCKET --prefix bucket/path/to/files/ | jq -r '.Contents[].Key'`

do

  echo $key

  aws s3 cp s3://$BUCKET/$key - | md5sum

Done

answered Mar 24, 2022 by gaurav
• 23,260 points

Related Questions In Others

0 votes
1 answer

How to create subnet inside a VPC in AWS using Terraform?

Hi@akhtar, You need to use aws_subnet resource inside your ...READ MORE

answered Jul 21, 2020 in Others by MD
• 95,440 points
425 views
0 votes
1 answer

How to use hexadecimal color strings in Flutter?

Hi@akhtar, In Flutter the Color class only accepts integers as parameters, or there ...READ MORE

answered Jul 24, 2020 in Others by MD
• 95,440 points
556 views
0 votes
1 answer

How to use Card Widget in Flutter App?

Hi@akhtar, A card is a sheet of material used to ...READ MORE

answered Aug 26, 2020 in Others by MD
• 95,440 points
944 views
0 votes
1 answer

How to use ListTile class in Flutter App?

Hi@akhtar, A list tile contains one to three ...READ MORE

answered Aug 26, 2020 in Others by MD
• 95,440 points
1,420 views
0 votes
1 answer

How to use for loop in Flutter?

Hi@akhtar, As the name suggests Dart for in ...READ MORE

answered Sep 3, 2020 in Others by MD
• 95,440 points
18,732 views
0 votes
2 answers

How to use the flat button in Flutter?

FlatButton is really easy to use. Where ever ...READ MORE

answered Nov 9, 2020 in Others by Vijay
• 200 points

edited Nov 9, 2020 by MD 3,768 views
0 votes
0 answers
0 votes
1 answer
0 votes
1 answer

NodeJS Amazon AWS S3 getObject how to send file in API response to download

Server Side const aws = require('aws-sdk'); router.get('/getfilefroms3', async (req, ...READ MORE

answered Mar 24, 2022 in Others by gaurav
• 23,260 points
9,096 views
0 votes
1 answer

How to open .xlsx files in MS Excel from VS Code?

Hello, to open xlxs files, or files ...READ MORE

answered Feb 17, 2022 in Others by gaurav
• 23,260 points
2,955 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP