AWS EMR script-runner access error

0 votes

I'm running emr-5.12.0, with Amazon 2.8.3, Hive 2.3.2, Hue 4.1.0, Livy 0.4.0, Spark 2.2.1 and Zeppelin 0.7.3 on 1 m4.large as my master node and 1 m4.large as core node.

I am trying to execute a bootstrap action that configures some parts of the cluster. One of these includes the line:

sudo sed -i '/zeppelin.pyspark.python/c\ \"zepplin.pyspark.python\" : \"python3\",' /etc/alternatives/zeppelin-conf/interpreter.json

It makes sure that the Zeppelin uses python3.4 instead of python2.7. It works fine if I execute this in the terminal after SSH'ing to the master node, but it fails when I submit it as a Custom JAR step on the AWS Web interface. I get the following error:

ed: can't read /etc/alternatives/zeppelin-conf/interpreter.json : No such file or directory Command exiting with ret '2'

The same thing happens if I use

sudo sed -i '/zeppelin.pyspark.python/c\ \"zepplin.pyspark.python\" : \"python3\",' /etc/zeppelin-conf/interpreter.json

Obviously I could just change it from the Zeppelin UI, but I would like to include it in the bootstrap action.


Aug 6, 2018 in AWS by bug_seeker
• 15,520 points

1 answer to this question.

0 votes

It turns out that a bootstrap action submitted through the AWS EMR web interface is submitted as a regular EMR step, so it's only run on the master node. This can be seen if you click the 'AWS CLI export' in the cluster web interface. The intended bootstrap action is listed as a regular step. Using the command line to launch a cluster with a bootstrap action bypasses this problem, so I've just used that.​

answered Aug 6, 2018 by Priyaj
• 58,090 points

Related Questions In AWS

0 votes
1 answer

A strange spark ERROR on AWS EMR

Those warning messages can be suppressed by ...READ MORE

answered Jul 13, 2018 in AWS by Flying geek
• 3,280 points
0 votes
1 answer

ERROR 1045 (28000): Access denied for user in AWS.

Hi@akhtar, You need to reset the password for this. ...READ MORE

answered Apr 2, 2020 in AWS by MD
• 95,440 points
0 votes
1 answer

"access denied" error when trying to move a file into the AWS server.

Hi@akhtar, I think you are trying to transfer file to ...READ MORE

answered Apr 2, 2020 in AWS by MD
• 95,440 points
+15 votes
2 answers

Git management technique when there are multiple customers and need multiple customization?

Consider this - In 'extended' Git-Flow, (Git-Multi-Flow, ...READ MORE

answered Mar 27, 2018 in DevOps & Agile by DragonLord999
• 8,450 points
+2 votes
1 answer
0 votes
1 answer

Getting memory leak error on AWS EMR - Saving Dataframe into hive external table on S3 -

You can start by creating  a temp ...READ MORE

answered Aug 9, 2018 in AWS by Priyaj
• 58,090 points
0 votes
1 answer

AWS Access Key error when uploading to S3

You can try getting creating a new ...READ MORE

answered Feb 6, 2019 in AWS by Priyaj
• 58,090 points
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP