Need help bootstrapping Python module installation on Amazon EMR

0 votes
Hi all, pretty simple question.

My requirement is that I need to make use of a Spark cluster through an EMR console. I will be running a Spark script that has the local dependency on a certain Python package.

What is the easiest way to go about doing this?

All help appreciated.
Feb 11, 2019 in Python by Anirudh
• 2,080 points

1 answer to this question.

0 votes


The easiest way to definitely do this is to create a bash script primarily. This script needs to contain your installation commands.

Later, you need to copy it to the S3 and set up a bootstrap action to point to the script (This is done from the console)

Consider the following example:

#!/bin/bash -xe

# Non-standard and non-Amazon Machine Image Python modules:
sudo pip install -U \
  awscli            \
  boto              \
  ciso8601          \
  ujson             \

sudo yum install -y python-psycopg2

Hope this helped!

answered Feb 11, 2019 by Nymeria
• 3,520 points

i tried but i got this error

sudo: easy_install-3.4: command not found
sudo: /usr/local/bin/pip3: command not found

Hello @ akash,

First of all: try pip3 instead of pip


pip3 --version
pip 9.0.1 from /usr/local/lib/python3.6/site-packages (python 3.6)

pip3 should be installed automatically together with Python3.x. The documentation hasn't been updated, so simply replace pip by pip3 in the instructions, when installing Flask for example.

Now, if this doesn't work, you might have to install pip separately.

Hope it helps!!
Thank you!

Related Questions In Python

0 votes
1 answer

Need help with Tkinter window formatting using Python

Tkininter comes with the columnspan argument to span the labels ...READ MORE

answered Sep 7, 2018 in Python by aryya
• 7,440 points
0 votes
1 answer

Need help extracting a schema to make use for an avro file in Python

Hi, nice question. So what I daily use ...READ MORE

answered Jan 10, 2019 in Python by Nymeria
• 3,520 points
–1 vote
1 answer
0 votes
1 answer

Need help with Python Text-to-Speech usage

Hi. Just before I begin my answer I ...READ MORE

answered Jan 16, 2019 in Python by Nymeria
• 3,520 points
0 votes
2 answers
+1 vote
2 answers

how can i count the items in a list?

Syntax :            list. count(value) Code: colors = ['red', 'green', ...READ MORE

answered Jul 6, 2019 in Python by Neha
• 330 points

edited Jul 8, 2019 by Kalgi 2,346 views
0 votes
0 answers
+5 votes
6 answers

Lowercase in Python

You can simply the built-in function in ...READ MORE

answered Apr 11, 2018 in Python by hemant
• 5,810 points
0 votes
1 answer

Need help installing easy_install in Python 2.7.1 on Windows 7

That tool is part of the setuptools ...READ MORE

answered Dec 26, 2018 in Python by Nymeria
• 3,520 points
+1 vote
1 answer

Need some help with Python memory leaks

As far as best practices, keep an ...READ MORE

answered Nov 26, 2018 in Python by Nymeria
• 3,520 points