AWS EMR script-runner access error

0 votes

I'm running emr-5.12.0, with Amazon 2.8.3, Hive 2.3.2, Hue 4.1.0, Livy 0.4.0, Spark 2.2.1 and Zeppelin 0.7.3 on 1 m4.large as my master node and 1 m4.large as core node.

I am trying to execute a bootstrap action that configures some parts of the cluster. One of these includes the line:

sudo sed -i '/zeppelin.pyspark.python/c\ \"zepplin.pyspark.python\" : \"python3\",' /etc/alternatives/zeppelin-conf/interpreter.json

It makes sure that the Zeppelin uses python3.4 instead of python2.7. It works fine if I execute this in the terminal after SSH'ing to the master node, but it fails when I submit it as a Custom JAR step on the AWS Web interface. I get the following error:

ed: can't read /etc/alternatives/zeppelin-conf/interpreter.json : No such file or directory Command exiting with ret '2'

The same thing happens if I use

sudo sed -i '/zeppelin.pyspark.python/c\ \"zepplin.pyspark.python\" : \"python3\",' /etc/zeppelin-conf/interpreter.json

Obviously I could just change it from the Zeppelin UI, but I would like to include it in the bootstrap action.

Thanks!

Aug 6, 2018 in AWS by bug_seeker
• 15,350 points
183 views

1 answer to this question.

0 votes

It turns out that a bootstrap action submitted through the AWS EMR web interface is submitted as a regular EMR step, so it's only run on the master node. This can be seen if you click the 'AWS CLI export' in the cluster web interface. The intended bootstrap action is listed as a regular step. Using the command line to launch a cluster with a bootstrap action bypasses this problem, so I've just used that.​

answered Aug 6, 2018 by Priyaj
• 56,900 points

Related Questions In AWS

0 votes
1 answer

A strange spark ERROR on AWS EMR

Those warning messages can be suppressed by ...READ MORE

answered Jul 13, 2018 in AWS by Flying geek
• 3,160 points
68 views
+1 vote
2 answers

How do I run python script on aws EC2 instance(ubuntu)?

I believe that you are using the ...READ MORE

answered Apr 17, 2018 in AWS by Cloud gunner
• 4,280 points
2,169 views
+1 vote
3 answers

Log in to AWS using Access Key ID and Secret Access Key ID

Access keys consist of an access key ...READ MORE

answered Aug 17, 2018 in AWS by Priyaj
• 56,900 points
1,055 views
+13 votes
2 answers

Git management technique when there are multiple customers and need multiple customization?

Consider this - In 'extended' Git-Flow, (Git-Multi-Flow, ...READ MORE

answered Mar 26, 2018 in DevOps & Agile by DragonLord999
• 8,380 points
178 views
0 votes
1 answer
0 votes
1 answer

Getting memory leak error on AWS EMR - Saving Dataframe into hive external table on S3 -

You can start by creating  a temp ...READ MORE

answered Aug 9, 2018 in AWS by Priyaj
• 56,900 points
162 views
0 votes
1 answer

AWS Access Key error when uploading to S3

You can try getting creating a new ...READ MORE

answered Feb 6 in AWS by Priyaj
• 56,900 points
1,302 views