Function of “hadoop namenode -format” command

0 votes

I am following the online tutorial to learn Hadoop and now I am installing Hadoop on my system. After configuring all the files in Hadoop. Now I am executing

hadoop namenode -format

After which I am getting the following message:

hadoop-1.2.1$ bin/hadoop namenode -format
14/11/26 12:37:16 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = localhost/127.0.0.1
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 1.2.1
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013

STARTUP_MSG:   java = 1.6.0_45

************************************************************/

14/11/26 12:37:17 INFO util.GSet: Computing capacity for map BlocksMap
14/11/26 12:37:17 INFO util.GSet: VM type       = 64-bit
14/11/26 12:37:17 INFO util.GSet: 2.0% max memory = 932118528
14/11/26 12:37:17 INFO util.GSet: capacity      = 2^21 = 2097152 entries
14/11/26 12:37:17 INFO util.GSet: recommended=2097152, actual=2097152
14/11/26 12:37:17 INFO namenode.FSNamesystem: fsOwner=vinay
14/11/26 12:37:17 INFO namenode.FSNamesystem: supergroup=supergroup
14/11/26 12:37:17 INFO namenode.FSNamesystem: isPermissionEnabled=true
14/11/26 12:37:17 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
14/11/26 12:37:17 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
14/11/26 12:37:17 INFO namenode.FSEditLog: dfs.namenode.edits.toleration.length = 0
14/11/26 12:37:17 INFO namenode.NameNode: Caching file names occuring more than 10 times 
14/11/26 12:37:17 INFO common.Storage: Image file /tmp/hadoop-vinay/dfs/name/current/fsimage of size 115 bytes saved in 0 seconds.
14/11/26 12:37:18 INFO namenode.FSEditLog: closing edit log: position=4, editlog=/tmp/hadoop-vinay/dfs/name/current/edits
14/11/26 12:37:18 INFO namenode.FSEditLog: close success: truncate to 4, editlog=/tmp/hadoop-vinay/dfs/name/current/edits
14/11/26 12:37:18 INFO common.Storage: Storage directory /tmp/hadoop-vinay/dfs/name has been successfully formatted.
14/11/26 12:37:18 INFO namenode.NameNode: SHUTDOWN_MSG: 

/************************************************************

SHUTDOWN_MSG: Shutting down NameNode at chaitanya-OptiPlex-3010/127.0.0.8

************************************************************/

Can anyone help me in understanding what is happening in this command. What are the changes which I will be getting after executing this command?

Can anyone help me in understanding what is happening in this command. What are the changes which I will be getting after executing this command.

Apr 15, 2018 in Big Data Hadoop by coldcode
• 2,010 points
3,271 views

2 answers to this question.

0 votes

Basically Hadoop namenode -format does following:

  • Fsimage files which contains filesystem images which is basically the metadata file in your NameNode. And edit logs are the files which contains the recent changes in the file system, which is later merged in the fsimage.  
  • Formatting the namenode deletes the information from namenode directory. The NameNode directory is specified in hdfs-site.xml file in dfs.namenode.name.dir property.
  • Formatting the file system means initializing the directory specified by the dfs.name.dir variable.
  • Never format, up and running Hadoop filesystem. You will lose all your data stored in the HDFS.  
answered Apr 15, 2018 by Shubham
• 13,250 points
0 votes

Namenode is the node in the Hadoop Distributed File System which keeps track of all the data stored in the Datanode. Namenode has metadata related to the data stored on the Datanodes and has information related to the location of the data stored. So, when you run the hadoop namenode -format command, all these information is deleted from the namenode which means that the system does not know where the data is stored hence losing all data stored in the Hadoop File System.

answered Feb 4 by Kanika

Related Questions In Big Data Hadoop

0 votes
1 answer
0 votes
1 answer

Hadoop: Format namenode gives error: Shutting down NameNode at java.net.UnknownHostException

UnknownHostException is thrown when hadoop tries to resolve ...READ MORE

answered Nov 12, 2018 in Big Data Hadoop by Omkar
• 67,160 points
207 views
0 votes
1 answer

Hadoop: namenode -format error "java.io.IOException"

Run the command as sudo or add the ...READ MORE

answered Nov 26, 2018 in Big Data Hadoop by Omkar
• 67,160 points
112 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,210 points
2,078 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
10,658 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
778 views
0 votes
1 answer
0 votes
1 answer

Best way of starting & stopping the Hadoop daemons with command line

First way is to use start-all.sh & ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by Shubham
• 13,250 points
1,050 views
+1 vote
3 answers

when i tried to run the hdfs namenode -format command...its failed to run..can anyone help me with this???

Hi Suriyaprakash, I guess the path for hadoop ...READ MORE

answered Jul 23, 2018 in Big Data Hadoop by avi_man_utd
342 views