AWS EMR script-runner access error

0 votes

I'm running emr-5.12.0, with Amazon 2.8.3, Hive 2.3.2, Hue 4.1.0, Livy 0.4.0, Spark 2.2.1 and Zeppelin 0.7.3 on 1 m4.large as my master node and 1 m4.large as core node.

I am trying to execute a bootstrap action that configures some parts of the cluster. One of these includes the line:

sudo sed -i '/zeppelin.pyspark.python/c\ \"zepplin.pyspark.python\" : \"python3\",' /etc/alternatives/zeppelin-conf/interpreter.json

It makes sure that the Zeppelin uses python3.4 instead of python2.7. It works fine if I execute this in the terminal after SSH'ing to the master node, but it fails when I submit it as a Custom JAR step on the AWS Web interface. I get the following error:

ed: can't read /etc/alternatives/zeppelin-conf/interpreter.json : No such file or directory Command exiting with ret '2'

The same thing happens if I use

sudo sed -i '/zeppelin.pyspark.python/c\ \"zepplin.pyspark.python\" : \"python3\",' /etc/zeppelin-conf/interpreter.json

Obviously I could just change it from the Zeppelin UI, but I would like to include it in the bootstrap action.

Thanks!

Aug 6, 2018 in AWS by bug_seeker
• 15,400 points
509 views

1 answer to this question.

0 votes

It turns out that a bootstrap action submitted through the AWS EMR web interface is submitted as a regular EMR step, so it's only run on the master node. This can be seen if you click the 'AWS CLI export' in the cluster web interface. The intended bootstrap action is listed as a regular step. Using the command line to launch a cluster with a bootstrap action bypasses this problem, so I've just used that.​

answered Aug 6, 2018 by Priyaj
• 57,550 points

Related Questions In AWS

0 votes
1 answer

A strange spark ERROR on AWS EMR

Those warning messages can be suppressed by ...READ MORE

answered Jul 13, 2018 in AWS by Flying geek
• 3,210 points
300 views
0 votes
1 answer

ERROR 1045 (28000): Access denied for user in AWS.

Hi@akhtar, You need to reset the password for this. ...READ MORE

answered Apr 2 in AWS by MD
• 41,340 points
745 views
0 votes
1 answer

"access denied" error when trying to move a file into the AWS server.

Hi@akhtar, I think you are trying to transfer file to ...READ MORE

answered Apr 2 in AWS by MD
• 41,340 points
182 views
+13 votes
2 answers

Git management technique when there are multiple customers and need multiple customization?

Consider this - In 'extended' Git-Flow, (Git-Multi-Flow, ...READ MORE

answered Mar 26, 2018 in DevOps & Agile by DragonLord999
• 8,420 points
598 views
0 votes
1 answer
0 votes
1 answer

Getting memory leak error on AWS EMR - Saving Dataframe into hive external table on S3 -

You can start by creating  a temp ...READ MORE

answered Aug 9, 2018 in AWS by Priyaj
• 57,550 points
344 views
0 votes
1 answer

AWS Access Key error when uploading to S3

You can try getting creating a new ...READ MORE

answered Feb 6, 2019 in AWS by Priyaj
• 57,550 points
2,886 views