What metadata is stored on a DataNode when a block is written to it?

0 votes
I am new to Hadoop and while learning about HDFS components, I read somewhere that some kind of metadata is stored on a DataNode of HDFS when a block is wriiten to it. So, what kind of metadata is stored on a DataNode?
Jul 23, 2018 in Big Data Hadoop by Meci Matt
• 9,420 points
229 views

1 answer to this question.

0 votes

Let me explain you step by step. 

Each DataNode keeps a small amount of metadata allowing it to identify the cluster it participates in. 

If this metadata is lost, then the DataNode cannot participate in an HDFS instance and the data blocks it stores cannot be reached. 

When an HDFS instance is formatted, the NameNode generates a unique namespace id for the instance. 

When DataNodes first connect to the NameNode, they bind to this namespace id and establish a unique “storage id” that identifies that particular DataNode in the HDFS instance. This data as well as information about what version of Hadoop was used to create the block files, is stored in a filed named VERSION in the ${dfs.data.dir}/active directory.

Hope it will answer to your query to some extent.

answered Jul 23, 2018 by nitinrawat895
• 10,710 points

Related Questions In Big Data Hadoop

0 votes
1 answer

What happens when a datanode that is dead becomes active again?

When NameNode notices that it has not ...READ MORE

answered Jun 20 in Big Data Hadoop by Rishi
103 views
0 votes
1 answer

Is there a way to rebalance single Datanode in Hadoop.

Currently Hadoop does not automatically do this. ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
72 views
0 votes
1 answer

What is Modeling data in Hadoop and how to do it?

I suggest spending some time with Apache ...READ MORE

answered Sep 19, 2018 in Big Data Hadoop by Frankie
• 9,810 points
109 views
0 votes
1 answer

Is it possible to start a zookeeper server instance in process?

To start ZooKeeper you have to execute ZooKeeperServerMain class. You can use ...READ MORE

answered Oct 9, 2018 in Big Data Hadoop by Omkar
• 67,660 points
41 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
1,196 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,710 points
3,323 views
0 votes
1 answer

How to get started with Hadoop?

Well, hadoop is actually a framework that ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by coldcode
• 2,020 points
103 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
16,408 views
0 votes
1 answer
0 votes
3 answers

What is Hive? Is Hive a database?

Hive is a data Warehouse infrastructure/system built ...READ MORE

answered Jul 1 in Big Data Hadoop by Ved Gupta
6,900 views