Terraform external data source formatting issue

0 votes

I want a list of my s3 buckets in TF.

With the AWS CLI I can get that list and format it into JSON:

my-mac$ eval aws s3 ls | awk '{print $3}' | jq -R -s -c 'split("\n")' | jq '.[:-1]'

[

  "xxx-bucket-1",

  "xxx-bucket-2"

]

But TF won't fetch it, and it's driving me mad.

Here's my data source config:

data "external" "bucket_objects" {

  program = ["bash", "cli.sh"]

}

The shell script file contents:

#!/bin/bash

set -e

eval aws s3 ls | awk '{print $3}' | jq -R -s -c 'split("\n")' | jq '.[:-1]'

And finally, the error message:

│ Error: Unexpected External Program Results

│ 

│   with data.external.bucket_objects,

│   on data.tf line 22, in data "external" "bucket_objects":

│   22:   program = ["bash", "cli.sh"]

│ 

│ The data source received unexpected results after executing the program.

│ 

│ Program output must be a JSON encoded map of string keys and string values.

│ 

│ If the error is unclear, the output can be viewed by enabling Terraform's

│ logging at TRACE level. Terraform documentation on logging:

│ https://www.terraform.io/internals/debugging

│ 

│ Program: /bin/bash

│ Result Error: json: cannot unmarshal array into Go value of type

│ map[string]string

My shell scripting game is weak and it's going to be something dumb, but I don't know what that is.

Dec 22, 2022 in AWS by Tejashwini
• 3,770 points
128 views

No answer to this question. Be the first to respond.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.

Related Questions In AWS

0 votes
1 answer

Issue with RDS import terraform

I guess this was the mistake you ...READ MORE

answered Nov 2, 2018 in AWS by Subhodeep Ghosh
1,600 views
0 votes
1 answer

Data source supported by AWS Glue?

The data source supported by AWS Glue ...READ MORE

answered Feb 4, 2019 in AWS by Heena
378 views
+3 votes
3 answers

Terraform AWS Cognito App Client

This feature is not currently supported by ...READ MORE

answered Aug 28, 2018 in AWS by eatcodesleeprepeat
• 4,710 points
2,421 views
0 votes
1 answer
0 votes
1 answer

Trying to upload data stream in Amazon S3

Try adding the ContentType & ACL in ...READ MORE

answered Jun 28, 2018 in AWS by Cloud gunner
• 4,660 points
1,925 views
0 votes
1 answer

AWS S3 uploading hidden files by default

versioning is enabled in your bucket. docs.aws.amazon.com/AmazonS3/latest/user-guide/….... the ...READ MORE

answered Oct 4, 2018 in AWS by Priyaj
• 58,100 points
3,986 views
–1 vote
1 answer

How to decrypt the encrypted S3 file using aws-encryption-cli --decrypt

Use command : aws s3 presign s3://mybucket/abc_count.png you get ...READ MORE

answered Oct 22, 2018 in AWS by Priyaj
• 58,100 points
4,035 views
0 votes
1 answer

Import my AWS credentials using python script

Using AWS Cli  Configure your IAM user then ...READ MORE

answered Nov 16, 2018 in AWS by Jino
• 5,810 points
2,113 views
0 votes
2 answers
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP