What metadata is stored on a DataNode when a block is written to it

0 votes
I am new to Hadoop and while learning about HDFS components, I read somewhere that some kind of metadata is stored on a DataNode of HDFS when a block is wriiten to it. So, what kind of metadata is stored on a DataNode?
Jul 23, 2018 in Big Data Hadoop by Meci Matt
• 9,460 points
1,567 views

1 answer to this question.

0 votes

Let me explain you step by step. 

Each DataNode keeps a small amount of metadata allowing it to identify the cluster it participates in. 

If this metadata is lost, then the DataNode cannot participate in an HDFS instance and the data blocks it stores cannot be reached. 

When an HDFS instance is formatted, the NameNode generates a unique namespace id for the instance. 

When DataNodes first connect to the NameNode, they bind to this namespace id and establish a unique “storage id” that identifies that particular DataNode in the HDFS instance. This data as well as information about what version of Hadoop was used to create the block files, is stored in a filed named VERSION in the ${dfs.data.dir}/active directory.

Hope it will answer to your query to some extent.

answered Jul 23, 2018 by nitinrawat895
• 11,380 points

Related Questions In Big Data Hadoop

0 votes
1 answer

What happens when a datanode that is dead becomes active again?

When NameNode notices that it has not ...READ MORE

answered Jun 20, 2019 in Big Data Hadoop by Rishi
2,700 views
0 votes
1 answer

Is there a way to rebalance single Datanode in Hadoop.

Currently Hadoop does not automatically do this. ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
280 views
0 votes
1 answer

What is Modeling data in Hadoop and how to do it?

I suggest spending some time with Apache ...READ MORE

answered Sep 19, 2018 in Big Data Hadoop by Frankie
• 9,830 points
1,069 views
0 votes
1 answer

Is it possible to start a zookeeper server instance in process?

To start ZooKeeper you have to execute ZooKeeperServerMain class. You can use ...READ MORE

answered Oct 9, 2018 in Big Data Hadoop by Omkar
• 69,190 points
198 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
3,300 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
8,604 views
0 votes
1 answer

How to get started with Hadoop?

Well, hadoop is actually a framework that ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by coldcode
• 2,080 points
519 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
75,814 views
0 votes
1 answer

What happens in a MapReduce job when you set the number of reducers to one?

If you set number of reducers as ...READ MORE

answered Jul 31, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
1,000 views
0 votes
3 answers

What is Hive? Is Hive a database?

Hive is a data Warehouse infrastructure/system built ...READ MORE

answered Jul 1, 2019 in Big Data Hadoop by Ved Gupta
18,675 views
webinar REGISTER FOR FREE WEBINAR X
Send OTP
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP