How to write examples to test whether the installation is validated or not

Question

I have successfully configured Hadoop 2.4 on my Ubuntu 14.04 using this tutorial.

http://dogdogfish.com/2014/04/26/installing-hadoop-2-4-on-ubuntu-14-04/

Now after completing installation how can I perform a test on it?

How and where can I get the test data or jar files?

Answer 1

You have some example jars in your hadoop installation directory.

The simplest thing you can do is run the teragen example(or wordcount). It is the first step in perform terasort.

Steps:

1. Go to the hadoop installation directory.
2. Run "hadoop jar hadoop-examples-0.20.2-cdh3u0.jar" to see all the jars you can run.
3. Go to home/[user] directory and create a file "example.txt" with the following data "This is a file to test Hadoop Installation example For the sake of the experiment, consider it to be 1TB"
4. While you are in that directory, run "hadoop dfs -put examples.txt /" this uploads the file onto your HDFS
5. Run "hadoop dfs -ls /" to check it is on there
6. Go to your Hadoop installation directory and run "hadoop jar hadoop-examples-0.20.2-cdh3u0.jar teragen 1000 /user/teragendata" - 1000 is the size data is to be broken into and the other param is the output directory.
7. On successful execution, you will see something like the text at the bottom.
8. Now to see how your MR job was run, in your browser open JobTracker and see the completed jobs. "localhost50030/jobtracker.jsp"

Hope this answer helps :)