Explain to me the difference between Name node and Secondary name node

Question

The Hadoop is considered to be unavailable when the name node is down even when it has a secondary name node.

Is it so that the Secondary name is not enough to back up Hadoop in such scenarios?

Can anyone help me to learn more about it?

Answer 1

First of all, let understand what is HDFS and its architecture in order to understand the solution to this question in a better way.

HDFS is a distributed file system which is deployed on low-cost commodity hardware with the following Architecture.

It is not as you predicted, Name node and Secondary name node are two different entities and they are designed for different tasks.

The name node is a master node that contains metadata in terms of FSimage and also contains the edit log. The edit log contains recently added/removed block information in the namespace of the name node. The FSimage file contains metadata of the entire Hadoop system in permanent storage. Every time we need to make changes permanently in FSimage, we need to restart name node so that edit log information can be written at name node, but it takes a lot of time to do that.

A secondary namenode is used to bring fsimage up to date. The secondary name node will access the edit log and make changes in FSimage permanently so that next time name node can start up faster.

Basically, the secondary name node is a helper for the name node and performs housekeeping functionality for the name node.