How to run a jar file in hadoop

0 votes
I have created a jar file using the java file
javac -classpath /usr/local/hadoop/hadoop-core-1.0.3.jar -d /home/user1/dir Dictionary.java
/usr/lib/jvm/jdk1.7.0_07/bin/jar cf Dictionary.jar /home/hduser/dir
Now i have tried running this jar with various commands
user1@ubuntu:~$ /usr/local/hadoop/bin/hadoop jar Dictionary.jar
Output:
Warning: $HADOOP_HOME is deprecated.

RunJar jarFile [mainClass] args...  
user1@ubuntu:~$ /usr/local/hadoop/bin/hadoop jar Dictionary.jar Dictionary
Output:
Warning: $HADOOP_HOME is deprecated.

Exception in thread "main" java.lang.ClassNotFoundException: Dictionary
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.hadoop.util.RunJar.main(RunJar.java:149)
How can i run the jar in hadoop?
Mar 26, 2018 in Big Data Hadoop by Shubham
• 13,490 points
25,485 views

7 answers to this question.

0 votes

Basically, the directory that you are packaging into the jar is confusing the jar file in locating the main class file.
Instead if trying do the following :
/usr/lib/jvm/jdk1.7.0_07/bin/jar cf Dictionary.jar /home/user1/dir/Dictionary.class
i.e. package the class file specifically into the jar and then run:
/usr/local/hadoop/bin/hadoop jar Dictionary.jar Dictionary
It just works fine provided that you have a main function in your class called Dictionary.
The problem is when you package a full directory inside a jar then the jar also needs to be aware of the directory structure to locate the class file. For this, we need to have a well defined package hierarchy to define the class location. So, when you are packaging /home/hduser/dir/ into the jar, the jar is not aware of the location of the class file which is located deep inside this directory structure. For this you need to add a package name to your .java file according to the directory structure , for example home.hduser.dir and while running the hadoop jar command specify the class name with the package structure, for example home.user1.dir.Dictionary. Learn more about Big Data Architect and its concepts from the Big data architect course.

answered Mar 26, 2018 by nitinrawat895
• 11,380 points
0 votes

You can try running jar from terminal like this:

hadoop jar <jarFileName> <mainClassname> <AnyCommandLineArguements>
answered Dec 10, 2018 by Namesh
0 votes
While creating the jar file, mention the output as .class file instead of .java file.
answered Dec 10, 2018 by Vilas
0 votes

As Vilas mentioned, jar file should be created as class. Here is what you can do:

javac -classpath /usr/local/hadoop/hadoop-core-1.0.3.jar -d /home/user1/dir Dictionary.class
answered Dec 10, 2018 by Sanket
0 votes

You can use the  RunJar() function to run your jar file:

RunJar(Dictionary.jar)
answered Dec 10, 2018 by Naveen
0 votes

Here's a syntax from the docs:

$ hadoop jar $HADOOP_HOME/hadoop-streaming.jar \
    -input myInputDirs \
    -output myOutputDir \
    -mapper org.apache.hadoop.mapred.lib.IdentityMapper \
    -reducer /bin/wc

Source: https://mapr.com/docs/52/ReferenceGuide/hadoop-jar.html

answered Dec 10, 2018 by Soukarod
0 votes

I used this command to run my wordcount jar. This could be helpful to you:

hadoop jar /opt/mapr/hadoop/hadoop-0.20.2/hadoop-0.20.2-dev-examples.jar wordcount /myvolume/in /myvolume/out
answered Dec 10, 2018 by Dasinto

Related Questions In Big Data Hadoop

0 votes
1 answer

Hadoop Pig: How to include external jar file in PIG?

You can do this: register /local/path/to/Jar_name.jar READ MORE

answered Nov 16, 2018 in Big Data Hadoop by Omkar
• 69,210 points
821 views
0 votes
1 answer

How to check the size of a file in Hadoop HDFS?

You can use the  hadoop fs -ls command to ...READ MORE

answered Nov 21, 2018 in Big Data Hadoop by Omkar
• 69,210 points
12,595 views
0 votes
1 answer

How to run Java file in Hadoop?

Try this: hadoop fs -cat hdfs:///user/hadoop/values.txt READ MORE

answered Dec 12, 2018 in Big Data Hadoop by Omkar
• 69,210 points
3,341 views
0 votes
1 answer

How to unzip a zipped file stored in Hadoop hdfs?

hadoop fs -text /hdfs-path-to-zipped-file.gz | hadoop fs ...READ MORE

answered Dec 12, 2018 in Big Data Hadoop by Omkar
• 69,210 points
11,769 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,234 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,523 views
0 votes
1 answer

How to get started with Hadoop?

Well, hadoop is actually a framework that ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by coldcode
• 2,080 points
889 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
103,823 views
0 votes
1 answer

How to delete a directory from Hadoop cluster which is having comma(,) in its name?

Just try the following command: hadoop fs -rm ...READ MORE

answered May 7, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
2,766 views
0 votes
1 answer

How to access different directories in a Hadoop cluster?

You need to configure the client to ...READ MORE

answered Sep 18, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
597 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP