How to run a jar file in hadoop?

0 votes
I have created a jar file using the java file
javac -classpath /usr/local/hadoop/hadoop-core-1.0.3.jar -d /home/user1/dir Dictionary.java
/usr/lib/jvm/jdk1.7.0_07/bin/jar cf Dictionary.jar /home/hduser/dir
Now i have tried running this jar with various commands
user1@ubuntu:~$ /usr/local/hadoop/bin/hadoop jar Dictionary.jar
Output:
Warning: $HADOOP_HOME is deprecated.

RunJar jarFile [mainClass] args...  
user1@ubuntu:~$ /usr/local/hadoop/bin/hadoop jar Dictionary.jar Dictionary
Output:
Warning: $HADOOP_HOME is deprecated.

Exception in thread "main" java.lang.ClassNotFoundException: Dictionary
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.hadoop.util.RunJar.main(RunJar.java:149)
How can i run the jar in hadoop?
Mar 26, 2018 in Big Data Hadoop by Shubham
• 12,790 points
3,594 views

7 answers to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes
Basically, the directory that you are packaging into the jar is confusing the jar file in locating the main class file.
Instead if trying do the following :
/usr/lib/jvm/jdk1.7.0_07/bin/jar cf Dictionary.jar /home/user1/dir/Dictionary.class
i.e. package the class file specifically into the jar and then run:
/usr/local/hadoop/bin/hadoop jar Dictionary.jar Dictionary
It just works fine provided that you have a main function in your class called Dictionary.
The problem is when you package a full directory inside a jar then the jar also needs to be aware of the directory structure to locate the class file. For this, we need to have a well defined package hierarchy to define the class location. So, when you are packaging /home/hduser/dir/ into the jar, the jar is not aware of the location of the class file which is located deep inside this directory structure. For this you need to add a package name to your .java file according to the directory structure , for example home.hduser.dir and while running the hadoop jar command specify the class name with the package structure, for example home.user1.dir.Dictionary.
answered Mar 26, 2018 by nitinrawat895
• 9,410 points
0 votes

You can try running jar from terminal like this:

hadoop jar <jarFileName> <mainClassname> <AnyCommandLineArguements>
answered Dec 10, 2018 by Namesh
0 votes
While creating the jar file, mention the output as .class file instead of .java file.
answered Dec 10, 2018 by Vilas
0 votes

As Vilas mentioned, jar file should be created as class. Here is what you can do:

javac -classpath /usr/local/hadoop/hadoop-core-1.0.3.jar -d /home/user1/dir Dictionary.class
answered Dec 10, 2018 by Sanket
0 votes

You can use the  RunJar() function to run your jar file:

RunJar(Dictionary.jar)
answered Dec 10, 2018 by Naveen
0 votes

Here's a syntax from the docs:

$ hadoop jar $HADOOP_HOME/hadoop-streaming.jar \
    -input myInputDirs \
    -output myOutputDir \
    -mapper org.apache.hadoop.mapred.lib.IdentityMapper \
    -reducer /bin/wc

Source: https://mapr.com/docs/52/ReferenceGuide/hadoop-jar.html

answered Dec 10, 2018 by Soukarod
0 votes

I used this command to run my wordcount jar. This could be helpful to you:

hadoop jar /opt/mapr/hadoop/hadoop-0.20.2/hadoop-0.20.2-dev-examples.jar wordcount /myvolume/in /myvolume/out
answered Dec 10, 2018 by Dasinto

Related Questions In Big Data Hadoop

0 votes
1 answer

Hadoop Pig: How to include external jar file in PIG?

You can do this: register /local/path/to/Jar_name.jar READ MORE

answered Nov 16, 2018 in Big Data Hadoop by Omkar
• 66,910 points
23 views
0 votes
1 answer

How to check the size of a file in Hadoop HDFS?

You can use the  hadoop fs -ls command to ...READ MORE

answered Nov 21, 2018 in Big Data Hadoop by Omkar
• 66,910 points
337 views
0 votes
1 answer

How to run Java file in Hadoop?

Try this: hadoop fs -cat hdfs:///user/hadoop/values.txt READ MORE

answered Dec 12, 2018 in Big Data Hadoop by Omkar
• 66,910 points
61 views
0 votes
1 answer

How to unzip a zipped file stored in Hadoop hdfs?

hadoop fs -text /hdfs-path-to-zipped-file.gz | hadoop fs ...READ MORE

answered Dec 12, 2018 in Big Data Hadoop by Omkar
• 66,910 points
1,143 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
655 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,410 points
1,843 views
0 votes
1 answer

How to get started with Hadoop?

Well, hadoop is actually a framework that ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by coldcode
• 2,010 points
52 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
9,123 views
0 votes
1 answer

How to access different directories in a Hadoop cluster?

You need to configure the client to ...READ MORE

answered Sep 18, 2018 in Big Data Hadoop by nitinrawat895
• 9,410 points
44 views
0 votes
1 answer

How can we transfer a PDF file to HDFS?

You can easily upload any file to ...READ MORE

answered Apr 13, 2018 in Big Data Hadoop by nitinrawat895
• 9,410 points
50 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.