How to call MapReduce program using a simple Java Program

Question

I have been trying to call a mapreduce job from a simple java program in the same package.. I tried to refer the mapreduce jar file in my java program and call it using the runJar(String args[]) method by also passing the input and output paths for the mapreduce job.. But the program dint work..

---

How do I run such a program where I just use pass input, output and jar path to its main method?? Is it possible to run a mapreduce job (jar) through it?? I want to do this because I want to run several mapreduce jobs one after another where my java program vl call each such job by referring its jar file.. If this gets possible, I might as well just use a simple servlet to do such calling and refer its output files for the graph purpose..

---

/*
 * To change this template, choose Tools | Templates
 * and open the template in the editor.
 */

/**
 *
 * @author root
 */
import org.apache.hadoop.util.RunJar;
import java.util.*;

public class callOther {

    public static void main(String args[])throws Throwable
    {

        ArrayList arg=new ArrayList();

        String output="/root/Desktp/output";

        arg.add("/root/NetBeansProjects/wordTool/dist/wordTool.jar");

        arg.add("/root/Desktop/input");
        arg.add(output);

        RunJar.main((String[])arg.toArray(new String[0]));

    }
}

Answer 1

Because map and reduce run on different machines, all your referenced classes and jars must move from machine to machine.

If you have package jar, and run on your desktop, @ThomasJungblut's answer is OK. But if you run in Eclipse, right-click your class and run, it doesn't work.

Instead of:

job.setJarByClass(Mapper.class);

Use:

job.setJar("build/libs/hdfs-javac-1.0.jar");

At the same time, your jar's manifest must include Main-Class property, which is your main class.

For Gradle users, can put these lines in build.gradle:

jar {
manifest {
    attributes("Main-Class": mainClassName)
}}