What metadata is stored on a DataNode when a block is written to it?

0 votes
I am new to Hadoop and while learning about HDFS components, I read somewhere that some kind of metadata is stored on a DataNode of HDFS when a block is wriiten to it. So, what kind of metadata is stored on a DataNode?
Jul 23, 2018 in Big Data Hadoop by Meci Matt
• 9,400 points
138 views

1 answer to this question.

0 votes

Let me explain you step by step. 

Each DataNode keeps a small amount of metadata allowing it to identify the cluster it participates in. 

If this metadata is lost, then the DataNode cannot participate in an HDFS instance and the data blocks it stores cannot be reached. 

When an HDFS instance is formatted, the NameNode generates a unique namespace id for the instance. 

When DataNodes first connect to the NameNode, they bind to this namespace id and establish a unique “storage id” that identifies that particular DataNode in the HDFS instance. This data as well as information about what version of Hadoop was used to create the block files, is stored in a filed named VERSION in the ${dfs.data.dir}/active directory.

Hope it will answer to your query to some extent.

answered Jul 23, 2018 by nitinrawat895
• 10,030 points

Related Questions In Big Data Hadoop

0 votes
1 answer

What happens when a datanode that is dead becomes active again?

When NameNode notices that it has not ...READ MORE

answered Jun 20 in Big Data Hadoop by Rishi
33 views
0 votes
1 answer

Is there a way to rebalance single Datanode in Hadoop.

Currently Hadoop does not automatically do this. ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
49 views
0 votes
1 answer

What is Modeling data in Hadoop and how to do it?

I suggest spending some time with Apache ...READ MORE

answered Sep 19, 2018 in Big Data Hadoop by Frankie
• 9,810 points
65 views
0 votes
1 answer

Is it possible to start a zookeeper server instance in process?

To start ZooKeeper you have to execute ZooKeeperServerMain class. You can use ...READ MORE

answered Oct 9, 2018 in Big Data Hadoop by Omkar
• 67,120 points
33 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
758 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,030 points
2,037 views
0 votes
1 answer

How to get started with Hadoop?

Well, hadoop is actually a framework that ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by coldcode
• 2,010 points
62 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
10,339 views
0 votes
1 answer
0 votes
3 answers

What is Hive? Is Hive a database?

Hive is a data Warehouse infrastructure/system built ...READ MORE

answered Jul 1 in Big Data Hadoop by Ved Gupta
4,430 views