Is there any way to make Apache Commons Math Library available to MapReducer job at runtime

Question

I need to perform statistical analysis in MapReduce job and would like to call methods in the Apache Commons Math library, which is distributed as a 1.3 megabyte Java archive (JAR) file. Is there any way to make this library available to MapReducer job at runtime?

Can someone please help?

Thanks in advance!

Answer 1

So, what you can do is when submitting the job on the command line, specify the –libjars option followed by the JAR file path.
 
The usage of the jar command is like this,
 
Usage: hadoop jar <jar> [mainClass] args...  
 
If you want the commons-math3.jar to be available for all the tasks you can do any one of these  

1. Copy the jar file in $HADOOP_HOME/lib dir  or  

2. Use the generic option -libjars.

Hope this helps!