Is there a way to install apache drill on an EMR cluster on AWS that is already Running?

+3 votes
Please Note I have a running EMR cluster that is persistent. I do not wish for a tear down.
Mar 26, 2018 in Cloud Computing by hemant
• 5,750 points
291 views

3 answers to this question.

0 votes

Try using Ruby Script. There one for AWS:

You should find one on Github. Make sure there is Ruby installed first. Also, I don't see a reason why it shouldn’t work on an existing cluster. EMR runs bootstrap scripts after all the selected EMR components are installed, Hadoop user created, and config settings applied. Of course, you'd test this on a new cluster, so you don't break your persistent one.

Make sure you use SCP or S3 because your script would run on every node. Also make sure you pick up the latest version to stay updated and installation to happen smoothly: https://drill.apache.org/download/

EMR clusters are bound to tear down. To ease this process, we have bootstrap scripts & cluster configuration. You may want to boot up, run job, tear down. This would be more affordable.

answered Mar 27, 2018 by brat_1
• 7,080 points
0 votes

Assuming you have SSM installed you can use ssm.send_command (send the command to all nodes) to call an S3 script which installs and starts drill on a running EMR cluster.

There is an example of this here: https://gist.github.com/fmcmac/a35738376d111fdca45057bd0fb4c79e

answered Aug 28, 2018 by eatcodesleeprepeat
• 4,670 points
0 votes

It looks to be trying to get apache-drill-1.0.0.tar.gz from http://getdrill.org/drill/download which should currently be v1.12.0 from the newer site https://drill.apache.org/download/ so at least that part needs to change. It also installs Zookeeper to run Drill, so check if you are already running Zookeeper as Drill will need to install as a new application in there if zookeeper is already there, you might experience issues if you try installing over the top, version compatibility is something to double check.

answered Oct 11, 2018 by findingbugs
• 4,750 points

Related Questions In Cloud Computing

+2 votes
2 answers

Is there a way to add cache control in AWS S3?

use of  current upstream master branch will ...READ MORE

answered Apr 10, 2018 in Cloud Computing by hemant
• 5,750 points
833 views
0 votes
1 answer

Is there a way to find out the number of objects stored in an S3 bucket?

Frankly, that is not possible. This is ...READ MORE

answered Apr 17, 2018 in Cloud Computing by code_ninja
• 6,220 points
34 views
0 votes
2 answers

Is there a way to copy AMI from one AWS account to another AWS account using boto?

You could share the AMI from one ...READ MORE

answered Aug 21, 2018 in Cloud Computing by Priyaj
• 56,520 points
300 views
0 votes
1 answer

AWS: Is there a way to simulate perimeter security using Amazon Web Services Model?

As we all know traditional perimeter security ...READ MORE

answered Jul 11, 2018 in Cloud Computing by DragonLord999
• 8,380 points
185 views
0 votes
1 answer

How to increase the EBS volume size of an instance that is on running state ?

Unluckily, it is not possible to increase ...READ MORE

answered Jul 17, 2018 in Cloud Computing by Gopalan
• 1,260 points
74 views
+2 votes
2 answers

Is there a way to get IAM Credentials from Cognito Username/Password from CLI?

Try doing it the other way round. ...READ MORE

answered Mar 27, 2018 in Cloud Computing by brat_1
• 7,080 points
53 views
0 votes
1 answer

Is there a way to import custom SVG shapes(clipart) directly into Dia?

If you are fine with compromising advanced ...READ MORE

answered Apr 18, 2018 in Cloud Computing by code_ninja
• 6,220 points
52 views
0 votes
1 answer

Is there a way to use amazon ECR image from one account in the code build of another account?

These forums might just about help https://forums.aws.amazon.com/t ...READ MORE

answered May 7, 2018 in Cloud Computing by brat_1
• 7,080 points
144 views
0 votes
1 answer

Is there a way to test codes that are written against AWS API?

Please note that you should not integrate ...READ MORE

answered Apr 17, 2018 in Cloud Computing by brat_1
• 7,080 points
19 views
0 votes
1 answer

Is there a way to transfer files between AWS s3 and ec2

The answer is a definite yes yes. ...READ MORE

answered Apr 17, 2018 in Cloud Computing by brat_1
• 7,080 points
49 views