Do we need to format datanode to store data

0 votes
Hi,

While setting up pseudo or full cluster, do we need to format datanode to install HDFS file system or we only have to format namenode.

I am asking this because, I have read this on many blogs that, HDFS stores the data in sequential order on the hard disk and it is an abstract layer which stores data on big blocks rather than default block size storage provided host file system.

If we are not formatting datanode than the powerful feature of HDFS comes to toss.
Jan 14 in Big Data Hadoop by Sam
• 160 points
103 views
Could you please elaborate what you meant by this statement:
"powerful feature of HDFS comes to toss."?

2 answers to this question.

0 votes

Hey @Sam!

Namenode is the component that holds all the records (metadata) of files stored in hdfs. It's holds the tree structure of all files stored on the cluster and keeps track of these files.

The common practice is to format only the namenode. When you format the namenode, the metadata related to the data stored in the hdfs is lost but in fact, the data is still present in the datanode. When the namenode is formatted, the namenode thinks that there is no data in the hdfs.

It is not a "must-do" step to format the datanode. Formatting the namenode will be just fine.

If you want to format the datanode, you can do it manually using the rmr option. Refer to the below syntax:

hadoop fs -rmr /path/to/hdfs
answered Jan 15 by Omkar
• 67,660 points
Hi,

Will there be any performance improvement in case of read and write operation if we format datanode explicitely.

Thanks
0 votes

That was not important. If you can answer, does  namenode -format command formats the datanode directory structure for HDFS file system for all the datanodes connected within cluster.

answered Jan 15 by Sam
• 160 points

The namenode -format command doesn't delete or format the datanode. The namenode -format only deletes the metadata on the namenode and formats the namenode. The data is still present on the datanodes but it can not be accessed because the after formatting the namenode, the namenode doesn't know where the data is present. 

Related Questions In Big Data Hadoop

0 votes
1 answer

I need to copy data from one HDFS to another HDFS. Can you help me do so?

I understood your issue. Let me help you ...READ MORE

answered May 16 in Big Data Hadoop by ravikiran
• 4,560 points
68 views
0 votes
1 answer

How can we send data from MongoDB to Hadoop?

The MongoDB Connector for Hadoop reads data ...READ MORE

answered Mar 26, 2018 in Big Data Hadoop by nitinrawat895
• 10,730 points
161 views
0 votes
1 answer

Can we use HDFS to store git repositories?

Yes, it is possible to use HDFS ...READ MORE

answered May 7, 2018 in Big Data Hadoop by nitinrawat895
• 10,730 points
162 views
0 votes
1 answer

What is Modeling data in Hadoop and how to do it?

I suggest spending some time with Apache ...READ MORE

answered Sep 19, 2018 in Big Data Hadoop by Frankie
• 9,810 points
114 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,730 points
3,371 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,730 points
405 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
16,739 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
1,222 views
0 votes
1 answer

How to create a parquet table in hive and store data in it from a hive table?

Please use the code attached below for ...READ MORE

answered Jan 28 in Big Data Hadoop by Omkar
• 67,660 points
3,064 views
0 votes
1 answer

Hadoop Hive: How to insert data in Hive table?

First, copy data into HDFS. Then create ...READ MORE

answered Nov 12, 2018 in Big Data Hadoop by Omkar
• 67,660 points
1,549 views