Is there a way to install apache drill on an EMR cluster on AWS that is already Running?

+3 votes
Please Note I have a running EMR cluster that is persistent. I do not wish for a tear down.
Mar 26, 2018 in Cloud Computing by hemant
• 5,750 points
228 views

3 answers to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

Try using Ruby Script. There one for AWS:

You should find one on Github. Make sure there is Ruby installed first. Also, I don't see a reason why it shouldn’t work on an existing cluster. EMR runs bootstrap scripts after all the selected EMR components are installed, Hadoop user created, and config settings applied. Of course, you'd test this on a new cluster, so you don't break your persistent one.

Make sure you use SCP or S3 because your script would run on every node. Also make sure you pick up the latest version to stay updated and installation to happen smoothly: https://drill.apache.org/download/

EMR clusters are bound to tear down. To ease this process, we have bootstrap scripts & cluster configuration. You may want to boot up, run job, tear down. This would be more affordable.

answered Mar 27, 2018 by brat_1
• 7,080 points
0 votes

Assuming you have SSM installed you can use ssm.send_command (send the command to all nodes) to call an S3 script which installs and starts drill on a running EMR cluster.

There is an example of this here: https://gist.github.com/fmcmac/a35738376d111fdca45057bd0fb4c79e

answered Aug 28, 2018 by eatcodesleeprepeat
• 4,670 points
0 votes

It looks to be trying to get apache-drill-1.0.0.tar.gz from http://getdrill.org/drill/download which should currently be v1.12.0 from the newer site https://drill.apache.org/download/ so at least that part needs to change. It also installs Zookeeper to run Drill, so check if you are already running Zookeeper as Drill will need to install as a new application in there if zookeeper is already there, you might experience issues if you try installing over the top, version compatibility is something to double check.

answered Oct 11, 2018 by findingbugs
• 4,730 points

Related Questions In Cloud Computing

+2 votes
2 answers

Is there a way to add cache control in AWS S3?

use of  current upstream master branch will ...READ MORE

answered Apr 10, 2018 in Cloud Computing by hemant
• 5,750 points
541 views
0 votes
1 answer

Is there a way to find out the number of objects stored in an S3 bucket?

Frankly, that is not possible. This is ...READ MORE

answered Apr 17, 2018 in Cloud Computing by code_ninja
• 5,920 points
25 views
0 votes
2 answers

Is there a way to copy AMI from one AWS account to another AWS account using boto?

You could share the AMI from one ...READ MORE

answered Aug 21, 2018 in Cloud Computing by Priyaj
• 56,140 points
207 views
0 votes
1 answer

AWS: Is there a way to simulate perimeter security using Amazon Web Services Model?

As we all know traditional perimeter security ...READ MORE

answered Jul 11, 2018 in Cloud Computing by DragonLord999
• 8,360 points
146 views
0 votes
1 answer

How to increase the EBS volume size of an instance that is on running state ?

Unluckily, it is not possible to increase ...READ MORE

answered Jul 17, 2018 in Cloud Computing by Gopalan
• 1,260 points
48 views
+2 votes
2 answers

Is there a way to get IAM Credentials from Cognito Username/Password from CLI?

Try doing it the other way round. ...READ MORE

answered Mar 27, 2018 in Cloud Computing by brat_1
• 7,080 points
34 views
0 votes
1 answer

Is there a way to import custom SVG shapes(clipart) directly into Dia?

If you are fine with compromising advanced ...READ MORE

answered Apr 18, 2018 in Cloud Computing by code_ninja
• 5,920 points
30 views
0 votes
1 answer

Is there a way to use amazon ECR image from one account in the code build of another account?

These forums might just about help https://forums.aws.amazon.com/t ...READ MORE

answered May 7, 2018 in Cloud Computing by brat_1
• 7,080 points
107 views
0 votes
1 answer

Is there a way to test codes that are written against AWS API?

Please note that you should not integrate ...READ MORE

answered Apr 17, 2018 in Cloud Computing by brat_1
• 7,080 points
12 views
0 votes
1 answer

Is there a way to transfer files between AWS s3 and ec2

The answer is a definite yes yes. ...READ MORE

answered Apr 17, 2018 in Cloud Computing by brat_1
• 7,080 points
35 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.