Is there a way to install apache drill on an EMR cluster on AWS that is already Running

+3 votes
Please Note I have a running EMR cluster that is persistent. I do not wish for a tear down.
Mar 27, 2018 in Cloud Computing by hemant
• 5,790 points
2,358 views

3 answers to this question.

0 votes

Try using Ruby Script. There one for AWS:

You should find one on Github. Make sure there is Ruby installed first. Also, I don't see a reason why it shouldn’t work on an existing cluster. EMR runs bootstrap scripts after all the selected EMR components are installed, Hadoop user created, and config settings applied. Of course, you'd test this on a new cluster, so you don't break your persistent one.

Make sure you use SCP or S3 because your script would run on every node. Also make sure you pick up the latest version to stay updated and installation to happen smoothly: https://drill.apache.org/download/

EMR clusters are bound to tear down. To ease this process, we have bootstrap scripts & cluster configuration. You may want to boot up, run job, tear down. This would be more affordable.

answered Mar 27, 2018 by brat_1
• 7,200 points
0 votes

Assuming you have SSM installed you can use ssm.send_command (send the command to all nodes) to call an S3 script which installs and starts drill on a running EMR cluster.

There is an example of this here: https://gist.github.com/fmcmac/a35738376d111fdca45057bd0fb4c79e

answered Aug 28, 2018 by eatcodesleeprepeat
• 4,710 points
0 votes

It looks to be trying to get apache-drill-1.0.0.tar.gz from http://getdrill.org/drill/download which should currently be v1.12.0 from the newer site https://drill.apache.org/download/ so at least that part needs to change. It also installs Zookeeper to run Drill, so check if you are already running Zookeeper as Drill will need to install as a new application in there if zookeeper is already there, you might experience issues if you try installing over the top, version compatibility is something to double check.

answered Oct 11, 2018 by findingbugs
• 4,780 points

Related Questions In Cloud Computing

+2 votes
2 answers

Is there a way to add cache control in AWS S3?

use of  current upstream master branch will ...READ MORE

answered Apr 10, 2018 in Cloud Computing by hemant
• 5,790 points
8,642 views
0 votes
1 answer

Is there a way to find out the number of objects stored in an S3 bucket?

Frankly, that is not possible. This is ...READ MORE

answered Apr 17, 2018 in Cloud Computing by code_ninja
• 6,300 points
854 views
0 votes
2 answers

Is there a way to copy AMI from one AWS account to another AWS account using boto?

You could share the AMI from one ...READ MORE

answered Aug 21, 2018 in Cloud Computing by Priyaj
• 58,100 points
2,200 views
0 votes
1 answer

AWS: Is there a way to simulate perimeter security using Amazon Web Services Model?

As we all know traditional perimeter security ...READ MORE

answered Jul 11, 2018 in Cloud Computing by DragonLord999
• 8,450 points
1,262 views
0 votes
1 answer

How to increase the EBS volume size of an instance that is on running state ?

Unluckily, it is not possible to increase ...READ MORE

answered Jul 17, 2018 in Cloud Computing by Gopalan
• 1,360 points
851 views
+2 votes
2 answers

Is there a way to get IAM Credentials from Cognito Username/Password from CLI?

Try doing it the other way round. ...READ MORE

answered Mar 27, 2018 in Cloud Computing by brat_1
• 7,200 points
1,232 views
0 votes
1 answer

Is there a way to import custom SVG shapes(clipart) directly into Dia?

If you are fine with compromising advanced ...READ MORE

answered Apr 18, 2018 in Cloud Computing by code_ninja
• 6,300 points
1,267 views
0 votes
1 answer

Is there a way to use amazon ECR image from one account in the code build of another account?

These forums might just help with AWS ...READ MORE

answered May 7, 2018 in Cloud Computing by brat_1
• 7,200 points
1,637 views
0 votes
1 answer

Is there a way to test codes that are written against AWS API?

Please note that you should not integrate ...READ MORE

answered Apr 17, 2018 in Cloud Computing by brat_1
• 7,200 points
541 views
0 votes
1 answer

Is there a way to transfer files between AWS s3 and ec2

The answer is a definite yes yes. ...READ MORE

answered Apr 17, 2018 in Cloud Computing by brat_1
• 7,200 points
844 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP