Function of hadoop namenode -format command

Question

I am following the online tutorial to learn Hadoop and now I am installing Hadoop on my system. After configuring all the files in Hadoop. Now I am executing

hadoop namenode -format

After which I am getting the following message:

hadoop-1.2.1$ bin/hadoop namenode -format

14/11/26 12:37:16 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = localhost/127.0.0.1
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 1.2.1
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013

STARTUP_MSG:   java = 1.6.0_45

************************************************************/

14/11/26 12:37:17 INFO util.GSet: Computing capacity for map BlocksMap
14/11/26 12:37:17 INFO util.GSet: VM type       = 64-bit
14/11/26 12:37:17 INFO util.GSet: 2.0% max memory = 932118528
14/11/26 12:37:17 INFO util.GSet: capacity      = 2^21 = 2097152 entries
14/11/26 12:37:17 INFO util.GSet: recommended=2097152, actual=2097152
14/11/26 12:37:17 INFO namenode.FSNamesystem: fsOwner=vinay
14/11/26 12:37:17 INFO namenode.FSNamesystem: supergroup=supergroup
14/11/26 12:37:17 INFO namenode.FSNamesystem: isPermissionEnabled=true
14/11/26 12:37:17 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
14/11/26 12:37:17 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
14/11/26 12:37:17 INFO namenode.FSEditLog: dfs.namenode.edits.toleration.length = 0
14/11/26 12:37:17 INFO namenode.NameNode: Caching file names occuring more than 10 times 
14/11/26 12:37:17 INFO common.Storage: Image file /tmp/hadoop-vinay/dfs/name/current/fsimage of size 115 bytes saved in 0 seconds.
14/11/26 12:37:18 INFO namenode.FSEditLog: closing edit log: position=4, editlog=/tmp/hadoop-vinay/dfs/name/current/edits
14/11/26 12:37:18 INFO namenode.FSEditLog: close success: truncate to 4, editlog=/tmp/hadoop-vinay/dfs/name/current/edits
14/11/26 12:37:18 INFO common.Storage: Storage directory /tmp/hadoop-vinay/dfs/name has been successfully formatted.
14/11/26 12:37:18 INFO namenode.NameNode: SHUTDOWN_MSG: 

/************************************************************

SHUTDOWN_MSG: Shutting down NameNode at chaitanya-OptiPlex-3010/127.0.0.8

************************************************************/

Can anyone help me in understanding what is happening in this command. What are the changes which I will be getting after executing this command?

Can anyone help me in understanding what is happening in this command. What are the changes which I will be getting after executing this command.

Answer 1

Basically Hadoop namenode -format does following:

• Fsimage files which contains filesystem images which is basically the metadata file in your NameNode. And edit logs are the files which contains the recent changes in the file system, which is later merged in the fsimage.  
• Formatting the namenode deletes the information from namenode directory. The NameNode directory is specified in hdfs-site.xml file in dfs.namenode.name.dir property.
• Formatting the file system means initializing the directory specified by the dfs.name.dir variable.
• Never format, up and running Hadoop filesystem. You will lose all your data stored in the HDFS.  

Answer 2

Namenode is the node in the Hadoop Distributed File System which keeps track of all the data stored in the Datanode. Namenode has metadata related to the data stored on the Datanodes and has information related to the location of the data stored. So, when you run the hadoop namenode -format command, all these information is deleted from the namenode which means that the system does not know where the data is stored hence losing all data stored in the Hadoop File System.