What metadata is stored on a DataNode when a block is written to it?

0 votes
I am new to Hadoop and while learning about HDFS components, I read somewhere that some kind of metadata is stored on a DataNode of HDFS when a block is wriiten to it. So, what kind of metadata is stored on a DataNode?
Jul 23, 2018 in Big Data Hadoop by Meci Matt
• 9,440 points
269 views

1 answer to this question.

0 votes

Let me explain you step by step. 

Each DataNode keeps a small amount of metadata allowing it to identify the cluster it participates in. 

If this metadata is lost, then the DataNode cannot participate in an HDFS instance and the data blocks it stores cannot be reached. 

When an HDFS instance is formatted, the NameNode generates a unique namespace id for the instance. 

When DataNodes first connect to the NameNode, they bind to this namespace id and establish a unique “storage id” that identifies that particular DataNode in the HDFS instance. This data as well as information about what version of Hadoop was used to create the block files, is stored in a filed named VERSION in the ${dfs.data.dir}/active directory.

Hope it will answer to your query to some extent.

answered Jul 23, 2018 by nitinrawat895
• 10,760 points

Related Questions In Big Data Hadoop

0 votes
1 answer

What happens when a datanode that is dead becomes active again?

When NameNode notices that it has not ...READ MORE

answered Jun 20 in Big Data Hadoop by Rishi
153 views
0 votes
1 answer

Is there a way to rebalance single Datanode in Hadoop.

Currently Hadoop does not automatically do this. ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
78 views
0 votes
1 answer

What is Modeling data in Hadoop and how to do it?

I suggest spending some time with Apache ...READ MORE

answered Sep 19, 2018 in Big Data Hadoop by Frankie
• 9,810 points
129 views
0 votes
1 answer

Is it possible to start a zookeeper server instance in process?

To start ZooKeeper you have to execute ZooKeeperServerMain class. You can use ...READ MORE

answered Oct 9, 2018 in Big Data Hadoop by Omkar
• 68,160 points
45 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
1,322 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,760 points
3,561 views
0 votes
1 answer

How to get started with Hadoop?

Well, hadoop is actually a framework that ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by coldcode
• 2,040 points
114 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
18,278 views
0 votes
1 answer
0 votes
3 answers

What is Hive? Is Hive a database?

Hive is a data Warehouse infrastructure/system built ...READ MORE

answered Jul 1 in Big Data Hadoop by Ved Gupta
7,598 views