Do we need to format datanode to store data

0 votes
Hi,

While setting up pseudo or full cluster, do we need to format datanode to install HDFS file system or we only have to format namenode.

I am asking this because, I have read this on many blogs that, HDFS stores the data in sequential order on the hard disk and it is an abstract layer which stores data on big blocks rather than default block size storage provided host file system.

If we are not formatting datanode than the powerful feature of HDFS comes to toss.
Jan 15, 2019 in Big Data Hadoop by Sam
• 160 points
3,852 views
Could you please elaborate what you meant by this statement:
"powerful feature of HDFS comes to toss."?

2 answers to this question.

0 votes

Hey @Sam!

Namenode is the component that holds all the records (metadata) of files stored in hdfs. It's holds the tree structure of all files stored on the cluster and keeps track of these files.

The common practice is to format only the namenode. When you format the namenode, the metadata related to the data stored in the hdfs is lost but in fact, the data is still present in the datanode. When the namenode is formatted, the namenode thinks that there is no data in the hdfs.

It is not a "must-do" step to format the datanode. Formatting the namenode will be just fine.

If you want to format the datanode, you can do it manually using the rmr option. Refer to the below syntax:

hadoop fs -rmr /path/to/hdfs
answered Jan 15, 2019 by Omkar
• 69,230 points
Hi,

Will there be any performance improvement in case of read and write operation if we format datanode explicitely.

Thanks
0 votes

That was not important. If you can answer, does  namenode -format command formats the datanode directory structure for HDFS file system for all the datanodes connected within cluster.

answered Jan 16, 2019 by Sam
• 160 points

The namenode -format command doesn't delete or format the datanode. The namenode -format only deletes the metadata on the namenode and formats the namenode. The data is still present on the datanodes but it can not be accessed because the after formatting the namenode, the namenode doesn't know where the data is present. 

Related Questions In Big Data Hadoop

0 votes
1 answer

I need to copy data from one HDFS to another HDFS. Can you help me do so?

I understood your issue. Let me help you ...READ MORE

answered May 16, 2019 in Big Data Hadoop by ravikiran
• 4,620 points
2,450 views
0 votes
1 answer

How can we send data from MongoDB to Hadoop?

The MongoDB Connector for Hadoop reads data ...READ MORE

answered Mar 27, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
2,063 views
0 votes
1 answer

Can we use HDFS to store git repositories?

Yes, it is possible to use HDFS ...READ MORE

answered May 7, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
1,384 views
0 votes
1 answer

What is Modeling data in Hadoop and how to do it?

I suggest spending some time with Apache ...READ MORE

answered Sep 19, 2018 in Big Data Hadoop by Frankie
• 9,830 points
1,743 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,927 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,454 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
108,322 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,547 views
0 votes
1 answer

How to create a parquet table in hive and store data in it from a hive table?

Please use the code attached below for ...READ MORE

answered Jan 28, 2019 in Big Data Hadoop by Omkar
• 69,230 points
18,560 views
0 votes
1 answer

Hadoop Hive: How to insert data in Hive table?

First, copy data into HDFS. Then create ...READ MORE

answered Nov 12, 2018 in Big Data Hadoop by Omkar
• 69,230 points
9,739 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP